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Abstract 

This  research  evaluated  the  effectiveness  of  the  enlisted  evaluation  performance 
feedback  system.  The  Air  Force  created  the  feedback  system  because  many  airmen, 
especially  those  in  lower  ranks,  did  not  know  or  did  not  understand  their  duties. 
Therefore,  fw  the  purpose  of  this  study,  effectiveness  was  defined  as  the  degree  to 
which  the  Air  Force  model  mimicked  the  ideal  feedback  model  as  formulated  by  the 
researchers’  analysis  of  the  literatiu^.  The  researchers  found  evidence  that  the  new  Air 
Force  feedback  system  is  an  improvement  over  the  old  design.  Under  the  old  system, 
the  only  regulated  means  of  providing  feedback  was  a  formal  report  which  was  issued, 
on  the  average,  once  a  year.  Since  it  was  a  formal  rating,  it  was  subject  to  several 
problems,  not  the  least  of  which  was  a  tendency  towards  inflationary  ratings.  These 
problems  made  the  system  somewhat  ineffective  with  respect  to  making  it  a  useful 
tool  for  providing  feedback  to  airmen  and  NCXDs.  The  new  procedure,  with  its 
regulated  informal  structure,  is  much  more  efficient  at  providing  accurate  and  timely 
feedback. 
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AN  ANALYSIS  OF  THE  AIR  FORCE  ENLISTED 
PERFORMANCE  FEEDBACK  SYSTEM 


I.  Introduction 


General  Issue. 

An  effective  performance  appraisal  system  can  significantly  contribute  to  an 
organization’s  productivity  (Lee,  1989:91).  Therefore,  it  is  in  the  best  interest  of  the 
Air  Force  to  have  an  effective  performance  appraisal  system.  From  a  study  conducted 
by  the  Air  Force  Military  Personnel  Center  in  1988,  the  Air  Force  concluded  that  the 
enlisted  evaluation  system  (EES)  was  ineffective  (Dept  USAF,  1990:167).  In  an  effon 
to  increase  the  effectiveness  of  that  system,  the  Air  Force  created  a  performance 
feedback  procedure. 

The  Air  Force  uses  a  four  step  process  to  evaluate  airmen  and  NCOs.  The  first 
part  of  this  procedure  consists  of  observing.  The  supervisor  examines  any  and  all 
aspects  of  a  subordinate’s  performance.  An  individual’s  bearing,  behavior,  adherence 
to  standards,  performance  of  duties,  and  quality  of  work  are  all  closely  scrutinized 
during  this  period  (Dept  USAF,  1990:166).  The  second  part  of  the  rating  process  is 
the  actual  evaluation  phase  in  which  the  rater  evaluates  the  performance  of  his  or  her 
subordinate  against  specific  standards  on  the  performance  feedback  worksheet  (PFW) 
and  enlisted  performance  report  (EPR)  (Dept  USAF,  1990:166).  The  supervisor  then 
provides  feedback  to  his  or  her  subordinate  using  the  PFW  as  a  tool  (note  that  the 
final  phase  consists  of  the  rater  merely  recording  what  was  obser .  1). 

Feedback  must  be  provided  to  the  ratee  by  his  or  her  supervisor  within  30  days 
of  having  been  assigned  to  that  rater.  This  performance  appraisal  is  known  as  the 
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initial  feedback  session  (Dept  USAF,  1990:167).  This  first  session  is  designed  to  let 
the  airman  or  NCO  being  appraised  know  exactly  what  kind  of  performance  the  rater 
expects  from  him  or  her.  Roughly  180  days  after  the  initial  feedback  session  is 
conducted,  a  supervisor  must  conduct  a  second  performance  appraisal.  This  second 
meeting,  known  as  the  follow-up  feedback  session,  has  two  primary  purposes.  First,  it 
allows  the  rater  to  discuss  the  level  of  performance  that  the  ratee  demonstrated  from 
the  initial  feedback  session  to  the  follow-up  session.  Second,  it  allows  the  rater  to  tell 
the  subordinate  what  his  or  her  (the  rater)  future  expectations  are  with  respect  to  that 
level  of  performance  (Dept  USAF,  1990:167),  A  supervisor  is  also  required  to 
provide  follow-up  feedback  within  30  days  of  completing  an  EPR  (EPRs  are 
completed  on  an  annual  basis).  As  can  be  seen,  an  individual  who  is  newly  assigned 
to  a  unit  will  receive,  at  a  minimum,  three  performance  feedback  sessions  within  one 
year  of  anriving  at  this  new  duty  station.  It  should  also  be  noted  that  performance 
feedback  was  added  in  1988  as  a  result  of  the  revisions  made  to  the  EES  (Dept  USAF, 
1990:166).  Since  the  implementation  of  this  innovative  program,  no  formal  studies 
have  been  conducted  on  the  effectiveness  of  the  new  system. 

Problem  Statement. 

The  purpose  of  this  research  is  to  evaluate  the  effectiveness  of  the  enlisted 
evaluation  performance  feedback  system.  The  Air  Force  created  the  feedback  system 
because  many  airmen,  especially  those  in  lower  ranks,  did  not  know  or  did  not 
understand  their  duties.  Therefore,  for  the  purpose  of  this  study,  effectiveness  is 
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defined  as  the  degree  to  which  the  Air  Force  model  mimics  the  ideal  feedback  model 
as  formulated  by  the  researcher’s  analysis  of  the  literature. 


Definition  of  Terms. 

Table  1  provides  the  appropriate  acronyms  and  definitions  of  the  terms  used  in 
this  research: 


Table  1 

ACRONYMS  AND  DEFINITIONS  OF  VARIABLES 


ACRONYM  DEFINITION 

EPR  (Enlisted  Performance  Report)  An  EPR  is  an  official  record  of  the 

ratee’s  performance  provided  by  his  or  her  rater  (Dept  USAF, 
1990:172). 

EES  (Enlisted  Evaluation  System)  The  EES,  governed  by  AFR  39-62  and 

AFP  39-15,  establishes  the  method  in  which  raters  will  provide  written 
performance  feedback  and  official  performance  ratings  on  the 
individual(s)  they  rate  (Dept  USAF,  1990:166). 

PFW  (Performance  Feedback  Worksheet)  A  PFW  is  the  record  used  to 

document  the  written  communication  between  the  rater  and  ratee  about 
the  ratee’s  duty  performance  (Dept  USAF,  1990:167). 


Scope  and  Limitations. 

This  study  was  conducted  by  developing  what  could  be  considered  the  ideal 
personnel  performance  appraisal  feedback  model  based  on  the  literature.  Not  all 
factors  are  taken  into  consideration.  Rather,  the  most  critical  aspects,  as  pointed  out 
by  the  literature,  were  used  in  the  development  of  this  model. 
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Research  Hypothesis. 

The  Air  Force  performance  feedback  model  mimics  the  ideal  personnel  perfonnance 
appraisal  feedback  model. 

Summary  and  Overview. 

Since  an  effective  performance  appraisal  system  can  significantly  contribute  to 
an  organization’s  productivity,  the  Air  Force  should  try  to  ensure  they  have  an 
effective  performance  appraisal  system.  Towards  that  end,  the  Air  Force  recently 
introduced  an  official  performance  feedback  session  as  part  of  their  enlisted  evaluation 
process.  However,  to  date,  there  have  been  no  formal  studies  of  this  new  system. 

The  literature  review  which  follows  points  to  the  fact  that  many  civilian 
organizations  have  ineffective  systems.  Unfortunately,  these  same  studies  also 
demonstrated  that  an  effective  performance  appraisal  system  can  significantly  enhance 
the  productivity  of  an  organization.  In  addition,  these  studies  pointed  out  that  most  of 
the  researchers  were  in  agreement  as  to  which  factors  were  signiticant  when  it  came  to 
determining  the  effectiveness  of  feedback.  Of  tiiese  elements,  the  acceptance  of 
feedback  by  the  recipient  appeared  to  have  the  most  influence.  Through  this  literature 
review,  it  was  discovered  that  when  comparing  the  researcher’s  criteria  for  an  effective 
system  to  the  Air  Force  system,  the  enlisted  performance  feedback  procedure  should 
be  remailcably  effective,  if  it  is  being  implemented  as  designed  and  if  the  performance 
feedback  is  actually  being  carried  out  with  the  frequency  specified  by  Air  Force 
regulations.  Nevertheless,  the  Air  Force  is  not  a  civilian  organization,  and  the  fact 
their  personnel  iq>praisal  system  has  many  of  the  positive  attributes  of  civilian  systems 
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may  not  necessarily  mean  that  the  Air  Force  system  is,  in  fact,  effective.  In  order  to 
reach  some  sort  of  conclusion  to  that  end,  this  study  was  undertaken  by  the 
researchers. 
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11.  Literature  Review 


Introduction. 

This  literature  review  focuses  on  the  need  for  effective  performance  appraisal 
systems.  Performance  feedback  is  important  because  it  can  significantly  influence  the 
productivity  of  an  organization  (Lee,  1989:91).  In  fact,  some  researchers  claim  that 
reliable  and  timely  feedback  is  essential  to  preserving  elevated  levels  of  achievement 
(Keman  and  others,  1991:716).  This  particular  analysis  centers  on  studies  which  have 
been  conducted  on  the  implementation  of  feedback  systems  in  civilian  corporations. 

Three  key  areas  will  be  examined  in  this  literature  review.  Initially,  poorly 
implemented  systems  will  be  surveyed.  The  t^ceptance  of  feedback  will  be  explored 
next.  Third,  feedback  models  will  be  scrutinized.  Finally,  some  conclusions  will  be 
drawn  from  the  material,  and  a  need  for  further  study  will  be  recommended. 

Inadequate  Performance  Appraisal  Systems. 

Lee  points  out  many  corporations  have  performance  appraisal  systems,  but 
most  of  them  are  ineffective  (Lee,  1989:91).  In  fact,  some  systems  may  actually  hurt 
some  organizations.  On  the  other  hand,  good  performance  appraisal  systems  can 
enhance  an  employee’s  attitude  by  establishing  an  energetic  and  dynamic  atmosphere 
in  the  work  place  (Lee,  1989:91).  There  are  several  potential  problem  areas  pointed 
out  by  this  article.  An  organization  may  suffer  from  a  poorly  defined,  communicated, 
adapted,  supported,  or  monitored  system  (Lee,  1989:94). 


6 


Lee  suggests  an  organization  can  overcome  these  shortcomings  by  adhering  to 
several  guidelines.  Management  should  decide  what  kind  of  system  to  implement  and 
how  the  data  should  be  used.  Management  should  also  endorse  the  system  they 
implement  to  give  it  validity.  They  should  establish  education  programs  to  teach 
managers  and  employees  how  the  system  works.  The  system  must  be  flexible  enough 
to  be  used  by  every  facet  of  the  company.  The  system  must  remain  current. 

Therefore,  appraisals  must  be  given  on  a  regular  basis,  at  least  once  or  twice  a  year. 

To  be  effective,  a  conoborative  reviewer  should  be  assigned  to  each  performance 
discussion  to  verify  the  facts  of  the  appraisal.  The  appraisal  form  itself  should  reflect 
the  needs  of  the  company.  In  other  words,  it  should  reflect  the  organization  it 
represents,  rather  than  a  copy  of  an  appraisal  form  used  by  another  organization. 
Finally,  there  should  be  a  system  set  up  to  periodically  check  the  appraisal  system  to 
make  sure  it  is  functioning  correctly  (Lee,  1989:96). 

Phillips,  like  Lee,  contends  that  many  organizations  use  performance  appraisals 
but  few  evaluation  programs  work  well.  He  also  states  there  are  a  variety  of  danger 
signals  which  point  to  the  fact  an  organization’s  appraisal  system  is  not  functioning 
effectively.  One  of  the  more  recognizable  ones  is  employees  all  have  the  same 
performance  standards  regardless  of  their  cunent  level  of  responsibility,  current  level 
of  experience,  and  previous  level  of  performance  (Phillips,  1987:80). 

Similar  to  the  problem  of  like  performance  standards  is  that  all  employees  have 
the  same  end-of-year  rating.  From  this  audior’s  study,  he  found  that  most  employees, 
in  organizations  with  perfcnrmance  appraisal  systems,  receive  the  same  ranking 
(slightly  higher  than  an  average  rating).  Managers  also  tend  to  use  an  employee’s 
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previous  performance  report  to  draft  a  new  one.  As  a  result,  an  employee’s  record  of 
performance  may  appear  identical  throughout  his  or  her  career  (some  managers  change 
the  date  of  an  old  appraisal  and  resubmit  it  as  the  new  appraisal)  (Phillips,  1987:81). 

These  practices  lead  to  a  variety  of  problems.  Employees  in  this  situation 
believe  they  are  not  actually  receiving  feedback  on  their  performance.  In  fact,  if 
employees  and  managers  differ  signiticantly  on  the  date  when  feedback  was  given,  this 
can  also  be  a  danger  signal  the  performance  appraisal  system  is  ineffective.  Also, 
employees  may  be  promoted  or  carried  in  the  organization  on  the  basis  of 
performance  reports  that  are  invalid.  Phillips’  article  strongly  suggests  to  managers  to 
watch  out  for  these  danger  signals  and  take  appropriate  action  (Phillips,  1987:82). 

Another  danger  signal  can  be  seen  by  observing  the  way  a  company  delegates 
authority.  Delegation,  which  is  defined  as  allocating  and  authorizing  work  to 
employees,  is  a  key  aspect  of  management,  according  to  Longenecker.  However, 
despite  all  the  attention  this  facet  of  management  receives,  there  are  complaints  from 
subordinates  in  many  organizations  which  point  to  the  fact  that  delegation  is  not  being 
carried  out  effectively.  The  key  complaints  are  job  ambiguity,  lack  of  authority,  and 
infrequent  and  ineffective  feedback  (Longenecker,  1991:3). 

This  author  points  out  that  delegation  involves  three  separate  but  closely  related 
processes.  These  activities  consist  of  assigning  responsibilities,  granting  authority,  and 
providing  accountability.  The  author  also  points  out  these  activities  must  occur  on  a 
regular  basis.  In  other  words,  "it’s  not  a  one  shot  deal"  (Longenecker,  1991:3). 
Longenecker  also  put  forth  the  idea  all  of  these  activities  are  depen^nt  on  each  other. 
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If  any  of  the  parts  of  this  triangular  relationship  (responsibility,  authority,  and 
accountability)  fail,  the  entire  process  of  delegation  will  fail  (Longenecker,  1991:4). 

In  order  to  implement  effective  delegation  techniques,  the  author  had  the 
following  suggestions;  (1)  On  a  semi-annual  basis,  supervisors  should  compare  their 
ideas  of  their  subordinate’s  job  responsibilities  against  what  their  subordinates  think 
their  responsibilities  are;  (2)  supervisors  should  determine  the  extent  of  the  authority 
they  give  their  subordinates  (if  it  is  inadequate,  supervisors  could  be  limiting  the 
effectiveness  of  their  subordinates);  (3)  supervisors  should  provide  regular  feedback  to 
provide  accountability  and  monitor  progress  (Longenecker,  1991:5). 

Another  negative  attribute  of  a  poorly  implemented  or  designed  feedback 
system  is  that  it  may  cause  subordinates  to  perform  poorly  under  pressure  (Heaton  and 
Sigall,  1991:175).  Studies  have  indicated  that  the  greatest  amount  of  pressure  results 
when  an  individual  is  receiving  performance  feedback  in  a  simation  where  that 
individual  is  behind  in  their  work  and  one  mistake  could  result  in  a  failed  outcome 
(Baumeister  and  Steinhilber,  1984:90).  Feedback  that  is  ineffective  in  this  particular 
situation  will  often  cause  individuals  to  choke  under  pressure  (Heaton  and  Sigall, 
1991:176).  Also  when  a  supervisor  has  high  expectations  from  a  subordinate,  the  cost 
of  failure  is  higher  than  when  expectations  are  not  as  high  (Heaton  and  Sigall, 
1989:1020).  It  is  interesting  to  note  that  these  expectations,  although  high,  may  not 
specifically  require  success.  In  other  words,  a  favorable  outcome  is  not  always 
grounded  in  a  successful  outcome  from  the  perspective  of  some  supervisors  (Tjosvold, 
1985:372).  The  self-consciousness  of  the  individual  receiving  the  feedback  is  also  an 
important  consideration  with  respect  to  how  individuals  will  react  to  feedback  under 
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pressure.  According  to  a  study  conducted  by  Heaton  and  Sigall,  individuals  with  low 
self-consciousness  (people  who  focus  their  attention  outward,  towards  an  audience)  are 
more  likely  to  choke  when  the  chance  of  failure  in  front  of  the  evaluator  is  likely. 
Conversely,  individuals  with  high  self-consciousness  (people  who  focus  their  attention 
inward,  away  from  an  audience)  are  more  likely  to  choke  if  a  disapproving  self- 
construction  (self-construction  is  the  way  an  individual  views  himself  or  herself)  is 
looming  (Heaton  and  Sigall,  1991:185). 

The  degree  to  which  a  person’s  job  requires  them  to  work  with  other  people  in 
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a  group  may  also  have  a  significant  impact  on  the  effectiveness  of  performance 
feedback  (Harcum  and  Badura,  1990:629).  A  theory  called  the  second-chance 
hypothesis  (also  known  as  the  social-loafing  effect)  states  that  people  will  pace 
themselves  according  to  the  amount  of  work  they  are  required  to  do  in  the  amount  of 
time  they  have  to  do  it  in  (Harcum  and  Badura.  1990:630).  As  an  analogy,  a  runner 
who  had  to  run  ten  miles  would  not  run  at  full  speed  during  the  first  two  miles  of  the 
run.  Rather,  the  runner  would  pace  himself  or  herself  based  on  the  length  of  the  run 
and  his  or  her  ability  to  complete  the  run  in  a  given  time.  This  theory  goes  on  to  state 
that  people  will  tend  to  produce  at  a  lower  rate  when  working  in  group  activities 
because  they  perceive  it  is  only  necessary  to  contribute  a  minimal  amount  of  effort 
towards  the  group’s  goals  in  order  to  achieve  them.  Why  does  this  phenomenon 
occur?  According  to  the  second-chance  hypothesis,  typically,  it  is  one  individual 
within  a  group  that  amtributes  the  most  and  directs  the  group’s  activities  towards  their 
goals.  Hence,  the  other  individuals  in  the  group  feel  that  it  is  not  necessary  for  them 
to  contribute  to  any  great  degree  (Harcum  and  Badura,  1990:631).  This  problem  is 
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exacerbated  by  the  fact  that  the  combined  efforts  of  individuals  towards  a  common 
goal  often  obscures  the  individual  sacrifices.  Therefore,  it  is  difficult  to  accurately 
appraise  each  individual.  People  in  the  group  are  aware  of  this  fact,  and  it  also 
contributes  to  lower  productivity  from  each  individual  (hence  the  phrase,  social 
loafing)  (Szymanski  and  Harkins,  1987:891).  If  these  theories  are  true,  it  would  mean 
that  it  would  be  extremely  difficult  to  implement  an  effective  performance  evaluation 
system  for  individuals  where  group  projects  and  teamwork  were  a  significant  part  of 
their  job. 

Harcum  and  Badura  conducted  a  study  in  February  1990,  in  which  they  tested 
the  second-chance  (social  loafing)  hypothesis.  Specifically,  they  wanted  to  ascertain 
what  the  most  crucial  variables  were  with  respect  to  performance.  From  that 
experiment,  they  concluded  that  the  hypothesis  was  invalid.  Rather,  they  discovered 
that  the  existence  of  specific  and  rigorously  precise  job  performance  standards  had  a 
significantly  grei  impact  on  individuals  than  group  oriented  activities  (Harcum  and 
Badura,  1990:635).  In  other  words,  if  individuals  know  exactly  what  kind  of  work  is 
expected  from  them,  they  will  typically  strive  to  achieve  that  goal,  regardless  of  their 
function  within  a  group  setting.  It  is  critical  to  note  that  not  aU  studies  agree  with 
Harcum  and  Badura’s  study.  In  fact,  there  is  a  great  deal  of  inconsistency  between 
most  of  the  studies  that  have  been  done  in  this  area.  However,  most  researchers, 
including  Harcum  and  Badura,  concur  that  this  inconsistency  arises  from  the  fact  that 
environmental  and  situational  components  greatly  impact  the  effectiveness  of  job 
standard  perception  (Harcum  and  Badura,  1990:637). 
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Acceptance  of  Feedback. 

Employees  who  want  to  do  well,  according  to  Lawrie,  desire  clear  feedback  on 
a  frequent  basis.  Annual  formal  feedback  is  not  enough  because  it  becomes  part  of 
the  permanent  record  and  can  actually  be  less  valid  than  informal  appraisals  because 
of  that  fact  (Lawrie,  1989:22).  This  article  proposes  self-appraisal  as  a  more 
advantageous  tool  than  the  formal  appraisal.  Under  this  particular  concept,  employees 
are  asked  to  appraise  the  key  duties  they  have  performed  in  the  last  three  months, 
which  results  in  a  focus  on  concrete  and  current  performance,  rather  than  on  the  long 
term  and  abstract.  After  they  have  appraised  themselves,  these  answers  are  discussed 
with  a  trusted  associate  and  their  superior,  for  their  analysis  of  the  appraisal  (Lawrie, 
1989:22).  The  appraisal  is  also  compared  to  direct  reports.  After  all  the  comparisons 
have  been  made,  the  employee  looks  at  areas  of  convergence  to  confirm  the  reports 
validity.  Where  there  is  convergence  on  areas  requiring  improvement,  he  or  she  can 
implement  changes  to  that  effect  (Lawrie,  1989:23). 

One  of  the  advantages  to  this  system  is  the  fact  it  can  be  done  anytime  the 
employee  desires  feedback.  Since  this  feedback  process  is  informal,  it  does  not  have 
any  emotionally  charged  psychological  problems  associated  with  feedback  that 
becomes  part  of  an  individual’s  permanent  record.  Also,  because  of  the  focus  of  this 
type  of  self-appraisal,  it  can  be  a  powerful  indicator  of  an  individual’s  true 
performance  (Lawrie,  1989:23). 

Even  if  the  problem  of  what  kind  of  appraisal  system  to  implement,  such  as 
self-tqipraisal,  is  solved,  another  question  still  remains.  Who  should  receive 
performance  appraisals?  Should  feedback  only  be  given  to  employees  on  the  shop 
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floor?  This  is  not  the  case  according  to  Longenecker  and  Gioia.  These  authors 
pointed  out  organizations  in  the  United  States  lack  effectiveness,  and  many 
organizational  executives  are  in  large  part  to  blame  for  this  dilemma.  A  fundamental 
way  in  which  an  organization  can  improve  their  effectiveness  is  by  the  use  of  formal 
appraisals.  When  formal  appraisals  are  properly  implemented,  they  provide  people 
(including  managers)  a  chance  for  growth  and  motivation  (Longenecker  and  Gioia, 
1988:41).  Longenecker  and  Gioia  pointed  out  that  executives  receive  little  or  no 
feedback  from  their  supervisors,  and  when  they  do  receive  feedback,  it  is  usually 
inadequate.  From  the  extensive  interview  they  conducted  with  60  upper  level 
management  personnel,  the  authors  discovered  over  40  percent  had  not  received  a 
formal  appraisal  in  over  a  year  (Longenecker  and  Gioia,  1988:43).  Those  executives 
receiving  appraisals  described  them  as  infrequent,  irregular,  rushed,  and  informal. 
From  this  interview  they  also  concluded  that  managers  want  regular  and  formal 
feedback  on  their  performance.  In  fact,  these  executives  said  a  bad  appraisal  was 
better  than  no  appraisal  at  all  (Longenecker  and  Gioia,  1988:44). 

Longenecker  and  Gioia  have  come  up  with  some  beneficial  advice  to  help  the 
supervisors  of  upper  level  management  implenwnt  effective  appraisal  programs  with 
their  managers.  First,  supervisors  must  realize  appraisals  are  worthwhile.  Second, 
they  must  conduct  formal  appraisals.  Third,  they  must  avoid  the  common  pitfalls 
associated  with  formal  tqipraisal  feedback.  Finally,  they  should  use  ^praisals  in 
conjunction  with  the  performance  plan  (Longenecker  and  Gioia,  1988:46). 

The  degree  of  confidence  that  subordinates  have  in  the  feedback  system  itself 
is  also  viewed  by  some  researchers  as  a  critical  aspect  of  how  likely  they  (the 
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subordinates)  will  accept  feedback  (Orpen,  1991:1336).  Further  research  from  an 
experiment  that  was  conducted  by  Orpen  concurred  with  the  findings  of  Wexley  and 
Klimoski  (Orpen,  1991:1336).  From  a  list  of  ten  variables,  Orpen  found  a  significant 
statistical  correlation  between  corrfidence  and  existence  of  appraisal  systems,  frequency 
of  appraisals,  self-esteem,  and  internal  control  (Orpen,  1991:1337).  His  findings 
suggested  that  subordinates  desired  feedback  and  operated  mcne  effectively  when  they 
received  that  feedback  (Orpen,  1991:1337).  The  fact  that  the  subjects  in  this  study 
desired  feedback  on  a  frequent  basis  is  consistent  with  Longenecker’s  and  Gioia’s 
findings  (Lx>ngenecker  and  Gioia,  1988:44).  It  is  also  interesting  to  note  that  the 
reward  system  in  this  study  did  not  have  a  notable  impact  on  the  effectiveness  of  the 
feedback  system  (Orpen,  1991:1337).  In  other  words,  regardless  of  the  level  of 
compensation  the  subjects  in  this  experiment  received  for  their  work,  it  had  no 
statistically  significant  impact  on  the  way  they  accepted  feedback.  Also,  the  size  and 
structure  of  the  organization  in  this  experiment  yielded  statistically  insignificant  results 
with  respect  to  the  impact  of  a  feedback  system  (Orpen,  1991:1337). 

Because  of  his  findings  in  this  experiment,  Orpen  also  suggested  that  it  might 
be  in  the  best  interest  of  an  organization  to  raise  their  subordinates’  self-esteem  before 
administering  performance  feedback  to  them,  since  individuals  with  high  self-esteem 
are  more  likely  to  accept  feedback  (Orpen,  1991:1337).  These  findings  are  consistent 
with  research  conducted  by  Tang  (Tang  and  Sarsfield-Baldwin,  1991:567).  According 
to  the  results  of  an  experiment  conducted  by  Tang  and  Sarsfield-Baldwin,  self-esteem 
and  effort  were  directly  related  to  the  degree  to  which  feedback  was  effective  (Tang 
and  Sarsfield-Baldwin,  1991:571).  Tang  also  noted  that  even  individuals  with  low 
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work  ethics  could  be  made  more  productive  by  effective  feedback  (Tang,  1990:227). 

In  another  study  conducted  by  Tang,  he  found  that  those  individuals  with  a  low  work 
ethic  who  received  negative  feedback  showed  statistically  significant  signs  of  increased 
intrinsic  motivation  after  the  feedback  session  (Tang,  1990:227).  However,  it  should 
also  be  pointed  out  that  individuals  with  intermediate  or  high  work  ethics,  experienced 
increased  intrinsic  motivation  after  they  received  positive  feedback  rather  than  negative 
feedback  (Tang  and  Sarsfield-BaldAvin,  1991:571). 

The  manner  in  which  feedback  is  administered  is  also  a  critical  factor  when 
considering  the  degree  to  which  feedback  is  accepted.  A  case  in  point,  MacDonald 
found  that  although  many  corporations  provide  employees  the  avenue  of  self¬ 
development  through  feedback  in  the  form  of  performance  appraisals  from  assessment 
centers,  many  do  not  take  advantage  of  this  opportunity  (MacDonald,  1988:50). 

Steelcase  Inc.  employed  over  10,000  people  in  the  United  States.  This 
company  created  an  assessment  center  known  as  the  Identification  Development 
Program  (EDP).  Steelcase  compiled  data  from  299  employees  who  had  been  involved 
with  IDP  for  12  years  in  an  effort  to  evaluate  who  actually  used  self-development 
(MacDonald,  1988:50). 

From  their  findings,  they  discovered  some  interesting  phenomena.  The  most 
inqxntant  factor  in  determining  whether  an  employee  would  use  IDP  for  self¬ 
development  was  the  logic  of  the  recommendations  on  their  appraisal.  In  other  words, 
if  there  was  a  logical  correlation  between  the  recommendations  made  at  IDP  and  the 
talents  and  attributes  needing  refrnement,  the  subordinate  would  usually  accept  it.  The 
age  of  the  employee  was  also  a  critical  factor.  Employees  over  40  were  less  likely  to 
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heed  IDP  recommendations.  Those  workers  with  two  or  more  years  of  college  were 
more  likely  to  consider  the  advice  given  at  IDP,  suggesting  a  correlation  between 
education  and  acceptance.  Also,  those  desiring  to  attend  the  center,  were  more 
accepting  of  IDP’s  advice.  Employees  who  desired  promotion  or  believed  they  were 
promotable  were  more  likely  to  listen  to  IDP.  MacDonald  recommends  managers  take 
note  of  these  factors  when  assessing  the  recepdveness  of  employees  to  self¬ 
development  (MacDonald,  1988:51). 

Some  researchers  disagreed  with  MacDonald  on  his  findings  with  respect  to 
age  and  acceptance  of  feedback.  A  study  conducted  by  Shore  and  Bleicken  indicated 
that  the  age  of  the  recipient  was  not  a  critical  factor  with  regard  to  how  well  the 
recipient  would  receive  feedback  (Shore  and  Bleicken,  1991:1093).  They  concluded 
that  age  did,  in  fact,  play  a  role  in  the  type  of  evaluation  an  individual  would  receive, 
but  their  findings  were  tempered  by  the  fact  that  other  elements  were  predominantly 
responsible  for  the  type  of  rating  an  individual  received  (Shore  and  Bleicken, 
1991:1095).  As  an  example,  from  the  literature  Shore  and  Bleicken  analyzed,  they 
found  that  older  workers  tended  to  receive  lower  marks  for  interpersonal  skills  than 
younger  workers.  On  the  other  hand,  older  workers  tended  to  receive  higher  ratings 
for  self-development  than  younger  workers  (Shore  and  Bleicken,  1991:1095).  These 
ratings  were  also  effected  by  the  age  of  the  supervisors  who  made  the  evaluations. 
This  type  of  interaction  is  important  to  note  because  it  brings  emphasis  back  to 
MacDonald’s  point  that  individual’s  are  more  likely  to  accept  feedback  if  they  believe 
it  is  accurate.  Hence,  if  a  supervisor  knowingly  or  unknowingly  skews  the  data  from 
an  evaluation  because  of  a  subordinate’s  age,  they  may  be  less  likely  to  accept  it.  In 
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any  case,  Shore  and  Bleicken  believe  that  the  inconsistencies  between  the  way  a 
subordinate  rates  himself  or  herself  is  due  to  a  wide  range  of  variables  and  not  solely 
due  to  the  age  of  the  subordinate  (Shore  and  Bleicken,  1991:1103). 

Further  investigation  supports  MacDonald’s  and  Shore  and  Bleicken’s  research 
with  respect  to  the  strong  correlation  between  the  recipient’s  perception  that  the 
feedback  is  logical  and  the  acceptance  of  that  feedback  (Mikulincer,  1990:739). 
Mikulincer  also  believes  that  when  people  perceive  that  the  outcome  of  a  task  is 
consistent  with  the  outcome  of  similar  tasks  that  have  occurred  in  the  past,  they  will 
tend  to  look  for  the  same  attributes  for  the  current  task  as  Jiose  attributes  they 
perceived  in  the  past  (Mikulincer,  1990:740).  In  addition,  Mikulincer  found  a  strong 
correlation  between  how  well  a  person  perceives  they  understand  what  the  attributional 
characteristics  are  for  a  given  event  and  the  degree  to  which  they  think  the 
attributional  characteristics  of  the  event  effected  the  outcome  of  performing  the  task 
(Mikulincer,  1990:740).  For  those  individuals  that  either  did  not  have  a  strong 
understanding  of  the  attributional  characteristics  or  felt  that  the  attributional 
characteiisucs  did  not  have  a  significant  impact  on  the  outcome  of  the  event,  they 
tended  to  avoid  making  a  judgement  as  to  what  the  specific  causal  agents  were  that 
affected  the  outcome  of  the  event  (Mikulincer,  1990:740).  When  one  of  the  two 
sources  of  information  was  strong  and  the  other  weak  (for  example,  an  individual 
believed  they  had  a  firm  grasp  on  the  attributional  characteristics  which  affected  the 
event,  but  they  were  not  sure  whether  these  characteristics  had  a  strong  impact  on  the 
outcome  of  the  event),  they  were  likely  to  believe  the  information  from  the  stronger 
source  (Mikulincer,  1990:740).  However,  if  both  sources  of  information  were  strong 
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and  if  both  sources  agreed,  the  individual  was  very  likely  to  accept  the  feedback  as 
being  accurate.  Conversely,  if  both  sources  of  information  are  perceived  as  strong  but 
contradict  each  other,  the  person  will  choose  the  information  that  is  most  in  agreement 
with  their  psychology  (Mikulincer,  1990:740).  One  theory  suggests  that  people  with 
strong  personal  theories  in  this  situation  would  tend  to  disregard  the  data  they  receive 
and  make  a  judgement  on  the  degree  to  which  attributes  and  the  types  of  attributes 
affected  the  outcome  of  an  event,  based  on  their  particular  beliefs  alone  (Mikulincer, 
1990:740).  Seligman  and  others  found  that  most  individuals  preferred  to  use  the 
attributional  characteristics  for  a  given  event  to  determine  the  reasons  why  a  particular 
outcome  of  an  event  occurred.  It  was  only  when  the  attributional  characteristics  for  a 
given  event  were  unclear  that  individuals  relied  on  their  personal  theories  to 
determine  the  attributions  (Seligman  and  others,  1979:242).  Mikulincer’s  research 
supports  these  claims  which  state  that  individuals  are  more  likely  to  accept  feedback 
when  the  attributional  characteristics  of  that  feedback  are  made  clear  to  the  recipient 
(Mikulincer,  1990:749). 

Feedback  Models. 

Bannister  and  Balkin  have  created  a  theoretical  model  that  examines  the 
effectiveness  of  feedback  in  performance  evaluations  and  compensation  decisions.  The 
object  of  most  evaluation  and  compensation  systems  is  to  increase  performance  by 
increasing  motivation  (Bannister  and  Balkin,  1990:99).  One  major  process  that  occurs 
whenever  feedback  is  given  to  an  employee  is  that  both  the  source  and  the  recipient  of 
the  feedback  try  to  understand  exacdy  why  the  employee’s  performance  was  effective 
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or  ineffective  (Bannister  and  Balkin,  1990: 1(X)).  One  of  the  biggest  points  of  concern 
in  this  area  is  the  supervisor  tends  to  minimize  the  effect  of  situational  constraints  on 
the  employee’s  performance;  things  not  within  the  employee’s  control.  Conversely, 
the  employee  tends  to  exaggerate  the  situational  constraints  whenever  he  or  she 
receives  feedback,  especially  when  the  feedback  is  negative.  This  situation  increases 
the  likelihood  of  conflict  between  the  supervisor  and  subordinate.  It  makes  sense  then 
that  if  the  source  of  the  feedback  internalizes  (gives  credit  to  the  subordinate)  for 
effective  performance,  the  more  readily  the  feedback  will  be  accepted  by  the  receiver. 

It  is  also  reasonable  to  assume  that  if  the  source  of  the  feedback  contributes  ineffective 
performance  to  situational  constraints,  the  feedback  will  be  more  readily  accepted  by 
the  subordinate  (Bannister  and  Balkin,  1990:102).  Because  of  these  constraints, 
Bannister  and  Balkin  argue  for  a  separation  of  the  link  between  salary  adjustments  and 
performance  feedback  (Bannister  and  Balkin,  1990:105). 

It  is  also  interesting  to  note  that  there  seems  to  be  tradeoff  between  the 
specificity  or  exacmess  of  feedback  and  negativity  (Hogarth  and  others,  1991:749). 

As  the  specificity  of  feedback  increases,  subordinates  tend  to  learn  more.  However, 
with  this  corresponding  increase  in  exactness  comes  a  similar  degree  of  negativity,  and 
as  Bannister  and  Balkin  pointed  out,  the  more  negative  feedback  is,  the  less  likely  an 
individual  is  willing  to  accept  it  (Hogarth  and  others,  1991:749).  Research  conducted 
by  (Hogarth  and  others)  also  pointed  to  the  fact  that  people  may  believe  that  tiiey  have 
limited  control  over  their  environment,  and  hence,  they  feel  they  should  not  be  held 
responsible  for  those  areas  out  of  their  control  (Hogarth  and  others,  1991:750).  This 
conclusion  is  consistent  with  Bannister  and  Balkin’s  research.  Along  these  same  lines, 
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(Hogarth  and  others)  conducted  experiments  which  lent  credence  to  the  argument  that 
subordinates  learn  how  to  accomplish  their  job  at  a  faster  rate  if  they  know  exactly 
how  job  performance  (in  the  case  of  the  experiment,  decisions  and  the  outcomes  of 
those  decisions)  are  translated  into  a  score  on  a  performance  evaluation  instrument 
(Hogarth  and  others,  1991:750).  In  any  case,  the  degree  of  exacmess  is  a  key  factor  in 
any  performance  feedback  system  since  it  may  determine  how  efficiently  the 
individual  receiving  the  feedback  will  learn  (Hogarth  and  others,  1991:734). 

Another  team  of  researchers  attempted  to  create  a  more  comprehensive  model 
than  Bannister’s  and  Balkin’s  model.  An  experiment  was  conducted  by  DeGregorio 
and  Fisher  in  which  they  studied  four  different  models  of  appraisal  feedback  and  the 
responses  of  the  subjects  giving  and  receiving  the  feedback  from  the  various  methods 
(DeGregorio  and  Fisher,  1988:605).  The  four  types  of  techniques  they  studied  in  their 
analysis  included  no  feedback,  top-down  feedback,  simple  participative  feedback 
discussion,  and  private  self-appraisal  with  participative  discussion  (DeGregorio  and 
Fisher,  1988:607).  From  this  experiment,  several  key  statistical  trends  were 
discovered.  The  smallest  increase  in  productivity  occurred  in  those  participants 
receiving  no  feedback.  Subordinate  satisfaction  was  greatest  in  the  two  groups  which 
used  participative  feedback  methods  (DeGregorio  and  Fisher,  1988:614).  As  to  the 
authors’  key  hypothesis,  they  found  that  there  was  no  statistical  significance  between 
the  effectiveness  of  the  three  types  of  feedback  systems  used,  although  all  three 
feedback  systems  produced  better  results  than  the  no  feedback  method  (DeGregorio 
and  Fisher,  1988:615). 
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The  results  of  DeGregorio  and  Fisher’s  experiment  are  supported  by  another 
study  which  was  conducted  by  Somers  and  Bimbaum  (Somers  and  Bimbaum, 
1991:1081).  In  their  analysis,  Somers  and  Bimbaum  found  that  the  literature  indicated 
that  evaluation  methods  that  used  self-appraisal  were  not  very  useful  to  supervisors 
with  respect  to  measuring  subordinates  performance  (Somers  and  Bimbaum, 
1991:1081).  Although  some  researchers  believe  that  self-appraisals  can  help 
subordinates  with  personal  development,  few  believe  it  can  be  used  as  an  effective 
evaluation  tool  (Campbell  and  Lee,  1988:310).  This  conclusion  would  tend  to  give 
credence  to  DeGregorio  and  Fisher’s  argument  that  although  self-appraisals  are  more 
effective  than  no  appraisals  at  all,  they  are  not  as  effective  as  participative  feedback 
methods.  Somers  and  Bimbaum  believe,  however,  that  the  self-appraisal  method  may, 
in  fact,  be  a  valuable  evaluation  tool.  From  their  research,  they  concluded  that  most 
self-appraisal  programs  yield  poor  results  because  they  are  poorly  implemented 
(Somers  and  Bimbaum,  1991:1090).  They  also  believe  that  if  employees  are  trained 
on  how  to  use  a  self-appraisal  system  which  focuses  on  core  job  skills,  the  system 
might  be  an  effective  measurement  tool  (Somers  and  Bimbaum,  1991:1090). 

However,  they  also  listed  two  significant  problems  with  self-appraisal  systems  which 
were  not  resolved  by  their  study.  First,  in  the  particular  self-appraisal  system  they 
examined,  the  multi-trait  multi-method  matrix  (MTMM)  for  self-  and  supervisory 
ratings,  it  was  difficult  for  supervisOTS  to  measure  the  different  areas  of  job 
performance  independently  of  each  other  (Somers  and  Bimbaum,  1991:1089).  Second, 
Somers  and  Bimbaum  were  not  sure  if  it  was  statistically  possible  to  correct  the  halo 
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effect,  in  which  subordinates  rate  their  performance,  perhaps  unknowingly,  higher  than 
they  actually  deserve  (Somers  and  Bimbaum,  1991:1090). 

Regardless  of  the  varying  attributes  of  the  numerous  feedback  systems  which 
are  used,  the  simple  act  of  monitoring  performance  may  enhance  an  organization’s 
productivity  (Larson  and  Callahan,  1990:530).  Studies  conducted  by  Mintzberg  and 
Yukl  indicate  that  the  process  of  monitoring  performance  by  itself  does  not  increase 
productivity.  Rather,  it  is  the  actions  taken  towards  subordinates  by  supervisors  in 
response  to  what  the  raters  observe  that  contributes  to  the  productivity  of  an 
organization  (Larson  and  Callahan,  1990:530).  However,  in  a  study  conducted  by 
Larson  and  Callahan,  subjects  were  observed  performing  tasks  but  were  given 
absolutely  no  feedback.  Meanwhile,  the  researchers  recorded  the  subjects’ 
productivity  in  an  effort  to  ascertain  whether  the  simple  act  of  observing  increased 
productivity  (Larson  and  Callahan,  1990:536).  The  results  of  their  experiment 
demonstrated  a  significant  statistical  correlation  between  the  effects  of  monitoring 
work  and  the  productivity  of  the  individual’s  being  monitored  (Larson  and  Callahan, 
1990:536).  An  interesting  part  of  this  experiment  was  it  also  demonstrated  that  the 
people  performing  the  monitoring  function  of  this  experiment  could  increase  the 
productivity  of  subjects  in  a  certain  area  of  a  certain  task  by  varying  the  degree  of 
attention  they  (the  monitors)  gave  to  a  particular  area  (Larson  and  Callahan, 

1990:536).  Larson  and  Callahan  also  demonstrated  in  their  research  that  an  increase  in 
monitoring  will  not  necessarily  lead  to  an  increase  in  productivity.  In  fact,  they 
pointed  out  that  it  might  have  a  detrimental  impact.  Therefore,  the  key  to  effective 
monitoring  is  to  figure  out  how  much  is  necessary  to  induce  the  maximum  amount  of 
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productivity  (Larson  and  Callahan,  1990:536).  They  also  illustrated  that  studies  have 
indicated  that  this  optimal  degree  of  monitoring  will  differ  from  work  place  to  work 
place  and  will  vary  according  to  job,  environment,  and  attributes  of  the  individual 
being  noonitoied  (Komaki  and  others,  1989:527).  It  should  be  emphasized  that  their 
(Komaki)  research  did  not  endeavor  to  assess  die  impact  of  feedback  after  monitoring 
takes  place  (Larson  and  Callahan,  1990:536).  Hence,  the  reader  should  not  infer  that 
this  study  indicates  feedback  could  not  significantly  improve  productivity. 

Another  critical  aspect  of  the  effectiveness  of  a  feedback  model  concerns  the 
way  in  which  supervisors  react  to  subordinates’  performance.  According  to  numerous 
studies,  there  is  a  significant  relationship  between  a  supervisor’s  locus  of  control  and 
the  way  he  or  she  attributes  their  subordinates  performance  to  ability,  effort,  task 
difficulty,  or  luck  (Heneman  and  others,  1989:466).  (Locus  of  control  in  this  context 
refers  to  the  way  in  which  a  supervisor  relies  on  institutional  authority  to  manage  his 
or  her  subordinates).  Other  studies  have  also  shown  that  the  organizational  setting  has 
a  dramatic  impact  on  the  type  of  locus  of  control  a  supervisor  is  likely  to  have. 
(Ashkanasy,  1991:527).  Specifically,  Ashkanasy  found  that  supervisors  in  a  military 
environment  tended  to  have  a  more  external  locus  of  control  (Ashkanasy,  1991:528). 

In  other  words,  they  used  more  institutional  authcmty  to  manage  their  subordinates. 
Hence,  the  way  in  which  they  viewed  their  subcnxlinates’  performance  was  governed 
by  an  external  locus  of  control  outlook.  Ashkanasy  also  pointed  out  that  organizations 
which  rely  primarily  on  organizational  authority  typically  have  a  negative  attitude 
towards  the  perfmmance  of  their  subordinates  (Ashkanasy,  1991:528).  Also, 
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supervisors  operating  in  this  type  of  atmosphere,  judge  their  subordinates  more  harshly 
than  their  counterparts  who  work  in  organizations  which  foster  an  internal  locus  of 
control  (Tjosvold,  1985:373). 

Conclusion. 

After  examining  the  articles  on  performance  appraisal  feedback,  it  became  clear 
there  are  several  issues  on  which  the  authors  drew  the  same  conclusions.  Most  of  the 
authors  agreed  that  any  type  of  feedback  was  better  than  none  at  all.  Unfortunately, 
these  same  authors  pointed  out  the  fact  many  organizations  were  not  using  a  system  in 
dealing  with  their  personnel  (including  upper  level  management),  and  those  that  did 
employ  a  system,  did  it  so  infrequently  or  poorly  as  to  make  it  ineffective.  Most  of 
the  cures  needed  to  fix  ailing  systems  that  were  stressed  in  these  writings  revolved 
around  the  fact  that  organizations  must  first  realize  that  performance  appraisal 
feedback  can  be  an  effective  and  worthwhile  mol.  Most  of  the  authors  also  pointed  to 
the  fact  that  employees  desire  feedback,  which  is  another  reason  it  should  be 
accomplished. 

The  literature  also  focused  on  the  importance  of  an  effective  performance 
appraisal  feedback  system  because  of  the  impact  it  has  on  the  productivity  of  an 
organization.  Most  authors  agreed  that  when  the  right  system  is  being  utilized,  it  can 
significantly  increase  the  effectiveness  of  an  organization  (although  many  authors 
differed  on  what  they  thought  was  the  best  system). 

Despite  the  varying  systems  advocated  by  the  authors,  they  listed  pitfalls  which 
were  common  to  all  systems,  with  perhaps  the  most  frequent  mistake  being  feedback 
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is  not  accomplished  on  a  regular  basis.  It  also  seems  clear  that  whatever  system  is 
chosen,  its  design  should  fit  the  organization  and  the  people.  Many  of  the  authors  also 
pointed  out  that  the  recepdveness  of  the  employee  could  have  a  dramatic  impact  on 
whether  the  feedback  would  be  effective  or  not.  The  authors  used  a  wide  variety  of 
measures  to  test  receptivity,  from  the  age  of  the  employee  to  the  employee’s 
perception  of  the  accuracy  of  the  feedback.  Whatever  measure  the  authors  used,  they 
almost  all  agreed  that  the  more  receptive  the  employee  was  to  the  feedback,  the  more 
effective  the  system  was  likely  to  be.  Of  particular  importance  to  this  study  is  how 
these  findings  can  be  applied  to  the  enlisted  evaluation  system.  Although  no  formal 
studies  have  been  done  according  to  this  extensive  literature  review,  from  personal 
experience,  the  researchers  have  seen  situations  where  enlisted  performance  feedback 
was  not  accomplished.  If  this  is  true  across  the  Air  Force,  it  may  be  unfortunate 
because  it  seems  that  the  system  as  it  exists  mday  could  be  an  effective  one.  The 
system  has  many  of  the  positive  attributes  pronroted  by  the  literature.  For  one  thing,  it 
is  set  up  to  provide  feedback  several  times  a  year.  It  can  also  be  given  whenever  a 
subordinate  requests  feedback.  It  also  avoids  emotionally  charged  psychological 
problems  because  it  does  not  become  part  of  an  individual’s  permanent  record  (Dept 
USAF,  1990:168).  If  implemented  as  designed,  the  Air  Force  enlisted  performance 
feedback  system  can  provide  an  ainnan  or  NGO  with  infcnmation  on  exactly  how  well 
they  are  performing.  Also,  because  it  doesn’t  become  a  permanent  part  of  their 
record,  it  avoids  inflationary  evaluations. 


25 


III.  Ideal  Feedback  Model 


Introduction. 

As  mentioned  previously,  there  is  no  one  ideal  feedback  model  which  will  fit 
all  organizations.  However,  there  are  certain  independent  variables  which  are 
important  to  all  feedback  systems  (Lee,  1989:95).  In  this  chapter,  the  researchers  will 
constract  a  model  which  will  encompass  the  most  important  parameters,  common  to 
all  feedback  models.  The  model  will  contain  the  following  parameters:  demographic 
dissimilarities  of  recipients,  frequency  of  feedback,  management  suppon  of  program, 
appropriateness  of  system  with  respect  to  the  organization,  and  structural  barriers. 

Demographic  Dissimilarities  of  Recipients. 

Demographic  dissimilarities  of  recipients  has  a  twofold  meaning  in  this  model. 
The  first  refers  specifically  to  individuals  who  have  the  same  job  but  have  different 
backgrounds.  The  second  refers  to  people  who  have  different  jobs  (their  backgrounds 
may  be  the  same  or  varied  in  this  case).  Job  in  this  context  means  the  set  of  all  tasks 
that  must  be  performed  by  a  given  worker  (Chase  and  Aquilano,  1989:431).  Either  of 
the  above  two  situations  could  lend  themselves  to  being  die  least  favorable  situation  to 
achieving  an  effective  feedback  system  (MacDonald,  1988:51).  The  most  important 
factors,  with  respect  to  how  workers  may  differ,  are  as  follows:  formal  education, 
vocatkxial  or  technical  (job)  skills,  salary,  tenure,  and  age.  If  the  organization  has  a 
wide  variety  of  jobs  which  require  specialization,  a  position  factor  must  be  added  as 
well.  Consider  the  difficulty  of  performing  feedback  on  individuals  who  have  a 
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diversified  background  but  perform  the  exact  same  job  requiring  the  exact  same  skill. 
The  problem  stems  from  the  fact  that  the  feedback  system  must  be  extremely  dynamic 
since  it  must  deal  with  such  wide  vaiiadons  in  the  backgrounds  of  its  recipients. 
Another  important  point  to  consider  on  this  issue  is  that  demographic  variables  effect 
the  receptiveness  of  people  receiving  feedback.  As  an  example,  older  workers  who 
have  tenure  are  less  likely  to  be  receptive  to  feedback  than  younger  workers  that  do 
not  have  tenure  (MacE>onald,  1988:51).  Thus,  the  more  diversified  the  group  which  is 
receiving  the  feedback,  the  more  difficult  it  is  for  management  to  implement  an 
effective  feedback  system. 

A  more  desirable  situatio:.  'v  juld  be  one  in  which  at  least  some  of  the 
important  demographic  factors  of  each  recipient  receiving  feedback  would  be  the 
same.  For  example,  if  all  the  employees  in  a  specific  job  have  the  same  vocational 
skills,  this  demographic  factor  will  have  a  positive  impact  on  the  ease  of  administering 
feedback  effectively. 

Perhaps  the  most  desirable  situation  for  an  organization  to  be  in  is  one  in 
which  most  of  the  employees  have  the  same  job  and  the  same  background.  In 
addition,  it  would  be  desirable  for  the  recipients  to  have  background  factors  which  are 
amenable  to  receiving  feedback,  such  as  youth  for  example.  Since  the  employees  have 
similar  backgrounds  and  job  requirements  in  this  situation,  the  implementation  and 
administration  of  a  feedback  system  will  tend  to  be  much  easier  and  more  effective. 

Frequency  of  Feedback. 

Obviously,  in  organizations  which  have  no  feedback  system,  the  frequency  of 
feedback  is  nonexistent.  In  some  organizations,  feedback  is  performed  on  an  annual 


27 


basis  only  as  part  of  a  requirement  based  on  administrative  necessity  and  not  as  a 
mechanism  to  increase  the  effectiveness  of  the  organization  1989:92).  Since  the 
purpose  of  performing  feedback  in  this  organization  is  unrelated  to  executing  an 
effective  feedback  policy,  this  coiporadon’s  behavior  would  tend  to  mimic 
organizations  whose  frequency  of  feedback  is  zero. 

Some  organizations  perform  feedback  only  at  specified  intervals  but,  on 
average,  more  than  once  a  year.  Usually,  in  such  corporations,  a  conscious  attempt 
has  been  made  to  implement  an  effective  feedback  procedure.  However,  feedback  is 
rarely,  if  at  all,  performed  at  unscheduled  intervals. 

The  most  ideal  system  would  be  one  in  which  feedback  is  performed  as  often 
as  the  subordinate  needs  clarification  on  his  or  her  performance  (Lawrie,  1989:22).  In 
this  ideal  situation,  this  feedback  could  be  performed  at  the  request  of  the  recipient  or 
by  being  initiated  by  the  subordinate’s  supervisor. 

Management  Support  of  Program. 

In  some  organizations,  no  feedback,  formal  or  informal,  exists  at  all.  In  these 
cases,  it  can  be  said  that  the  leadership  of  the  corporation  gives  no  emphasis  to  a 
feedback  system,  since  it  has  not  bothood  to  implement  one  (Lee,  1989:95). 

Almost  at  this  same  extreme,  some  organizations  have  a  feedback  system,  but 
the  administration  of  the  program  is  not  enforced  by  the  leadership  within  the 
organization.  Therefore,  although  a  corporation  may  have  the  framework  of  system 
within  their  organizational  structure,  if  it  is  ignored  by  the  leadership  of  the 
corporation,  it  may  be  as  ineffective  as  those  OTganizations  that  have  no  system  at  all. 
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Hence,  this  classification  includes  organizations  which  do  not  have  a  system  and 
corporations  which  have  a  system  which  really  amounts  to  a  paperwork  exercise  where 
the  performance  appraisals  look  the  same  from  employee  to  employee  and  year  to 
year. 

A  more  moderate  approach  is  when  organizations  have  a  system  which  receives 
some  support  from  upper-level  management.  As  an  example,  some  corporations  have 
a  varying  degree  of  success  wdth  their  performance  appraisal  systems.  In  these 
organizations,  certain  areas  on  a  performance  report  may  be  valid  because  the 
leadership  within  the  organization  stresses  those  particular  domains.  For  example,  if 
management  stresses  that  certain  skills  be  accurately  recorded  for  purposes  of 
determining  who  should  receive  an  increase  in  pay,  the  feedback  system  in  that 
corporation  could  be  used  as  a  valuable  tool.  However,  if  this  is  the  extent  of  the 
emphasis,  the  feedback  system  may  be  lacking  in  other  areas  and  could  not  convey  an 
accurate  assessment  of  the  overall  performance  of  subordinates  (Lee,  1989:91). 

Those  feedback  systems  which  enjoy  the  greatest  support  from  upper-level 
management  are  usually  the  most  effective.  In  these  corporations,  top  management 
takes  an  active  role  in  ensuring  that  their  feedback  program  is  being  implemented  in 
accordance  with  organizational  goals.  Every  aspect  of  the  feedback  system  is  carefully 
monitored  by  the  leadership  within  the  corporation  for  proper  implementation,  strict 
cotr^liance,  and  flawless  accuracy.  In  these  orgaruzations,  decisions  relating  to 
promotion  and  position  are  often  heavily  correlated  with  the  feedback  system. 
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Appropriateness  of  System  with  Respect  to  the  Organization. 

How  well  a  particular  feedback  system  Hts  its  organizational  structure  is  critical 
to  determining  the  degree  of  effectiveness  the  system  will  achieve  (Lee,  1989:92).  At 
the  lowest  level  of  appropriateness,  an  organization  adopts  an  existing  system  from 
another  corporation  whose  structure  is  dissimilar  from  their  own,  making  little  or  no 
changes  to  the  system  after  they  adopt  it.  Obviously,  this  system  will  probably  have 
the  least  chance  of  success.  There  are  numerous  feedback  procedures,  all  of  which  are 
designed  for  a  particular  kind  of  organizational  layout.  The  closer  a  feedback  system 
fits  the  unique  characteristics  of  an  organization,  the  greater  the  likelihood  it  will  be 
successful. 

In  a  more  appropriate  approach,  an  organization  adopts  an  existing  system  from 
a  corporation  whose  structure  is  similar  to  their  own,  making  modifications  to  fit  their 
organization’s  particular  idiosyncracies.  In  this  moderate  approach,  an  effective 
system  can  be  implemented. 

At  the  highest  level  of  appropriateness,  an  organization  forms  a  task  force  to 
study  the  need  for  a  feedback  system.  The  task  force  creates  a  unique  feedback 
system,  custom-tailored  to  meet  the  specific  demands  of  their  organization.  Provided 
that  the  task  force  is  well-equipped  and  has  the  prerequisite  knowledge  on  how  to 
design  and  implement  a  feedback  system,  the  best  possible  design  for  an 
organizational  feedback  system  can  be  obtained  in  this  manner. 
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Structural  Barriers. 


Structural  barriers  refers  to  types  of  problems  an  organization  might  encounter 
because  of  the  difficulties  associated  with  the  type  of  feedback  system  they  have 
adopted.  Note  that  these  problems  do  not  necessarily  equate  with  obstacles  associated 
with  the  incompatibility  of  a  specific  system  with  a  certain  organization.  Rather,  these 
problems  are  inherent  with  the  particular  type  of  system  that  is  adopted,  regardless  of 
the  structure  of  the  organization.  For  example,  in  some  organizations,  a  formal 
feedback  system,  in  which  the  feedback  becomes  a  permanent  part  of  an  individual’s 
record,  is  the  sole  method  used  by  a  corporation  to  provide  feedback  to  its  recipients. 
As  described  earlier,  these  types  of  formal  reports  suffer  from  inflation  and  other 
problems  (Lawrie,  1989:22). 

Corporations  which  have  no  feedback  system  have  perhaps  the  largest 
structural  barrier  of  all  by  the  fact  that  they  have  no  feedback  system.  This  barrier  is 
usually  erected  by  the  leadership  of  an  organization  either  through  ignorance  or 
apathy. 

Some  organizations  have  feedback  systems  but  are  hindered  because  of  the 
feedback  procedures  they  are  using.  Some  examples  of  structural  barriers  are;  Reports 
may  be  infrequently  released.  The  reports  may  only  be  conducted  on  certain 
personnel.  The  feedback  may  be  limited  to  a  formal  report. 

A  more  fitting  model  would  be  one  in  which  all  or  most  of  the  structural 
barriers  were  removed  from  the  feedback  process  in  an  organization.  For  example,  if 
a  corporation  relied  solely  on  formal  feedback,  they  could  institute  an  informal 
feedback  procedure. 
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A  diagram  of  the  ideal  feedback  model  is  presented  next.  It  is  followed  by 
several  figures  which  list  the  various  model  "parameters"  with  examples  of  low, 
medium,  or  high  rankings. 
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Figure  1.  Ideal  Feedback  Model  Parameters 
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Table  2 


DEMOGRAPHIC  DISSIMILARITIES  OF  RECIPIENTS 
(BACKGROUND) 


Low 

Medium 

High 

There  is  complete  or  near 

There  are  two  to  three 

There  are  at  least  four 

complete  diversiEcation 

areas  that  are  similar  in 

areas  that  are  similar  in 

of  each  recipient’s 

each  recipient’s 

each  recipient’s 

background  with  respect 

background  with  respect 

background  with  respect 

to: 

to: 

to: 

1.  Formal  education 

1.  Formal  education 

1.  Formal  education 

2,  Technical  education 

2.  Technical  education 

2.  Technical  education 

3.  Salary 

3.  Salary 

3.  Salary 

4.  Tenure 

4.  Tenure 

4.  Tenure 

5.  Age 

5.  Age 

5.  Age 
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Table  3 


DEMOGRAPHIC  DISSIMILARITIES  OF  RECIPIENTS 

(JOB) 


Low 

Medium 

High 

There  are  a  great  number 
of  different  tasks 
requiring  a  high  degree  of 
specialization. 

There  are  a  variety  of 
jobs,  but  they  do  not 
require  a  great  deal  of 
specialization. 

-or- 

Many  of  the  jobs  are 
identical,  but  they  require 
as  great  deal  of 
specialization. 

All  or  most  jobs  are  the 
same  or  require  the  same 
degree  of  specialization. 
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Table  4 


FREQUENCY  OF  FEEDBACK 


Low 

Medium 

High 

Zero.  The  organization 
has  no  feedback  system. 

-or- 

Once  a  year.  Feedback  is 
performed  as  an 
administration  function 
without  regard  to 
increasing  the 
effectiveness  of  the 
organization. 

More  than  once  a  year. 
However,  feedback  is 
performed  only  at 
specified  intervals. 

Frequently.  Feedback  is 
performed  at  the  request 
of  the  recipient  or  by 
being  initiated  by  the 
subordinate’s  supervisor, 
as  well  as  at  specitied 
intervals. 

36 


Table  5 


MANAGEMENT  SUPPORT  OF  PROGRAM 


Low 

Medium 

High 

Program  does  not  exist. 
Hence,  leadership  gives 
no  emphasis  to  a  feedback 
system,  since  it  has  not 
bothered  to  implement 
one. 

-or- 

System  exists  in  an 
organization,  but 
administration  of  program 
is  not  enforced  by  the 
leadership  within  the 
organization. 

Program  receives  some 
support  from  upper-level 
management.  Certain 
areas  on  a  performance 
report  may  be  valid 
because  the  leadership 
within  the  organization 
stresses  those  particular 
domains. 

Program  receives  strong 
support  from  leadership. 
Top  numagement  takes  an 
active  role  in  ensuring 
that  their  feedback 
program  is  being 
implemented  in 
accordance  with 
organizational  goals. 
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Table  6 


APPROPRIATENESS  OF  SYSTEM  WITH  RESPECT  TO  ORGANIZATION 


Low 

Medium 

High 

Organization  adopts  an 
existing  feedback  system 
from  another  organization 
which  has  a  radically 
different  structure  from 
their  own.  In  addition, 
the  organization  makes  no 
changes  in  the  adopted 
system  to  make  it  a  more 
effective  instrument. 

Organization  adopts  an 
existing  feedback  system 
whose  structure  is  similar 
to  its  own,  making  some 
minor  changes  in  the 
adopted  system  in  an 
attempt  to  make  it  more 
effective. 

Organization  adopts  an 
existing  feedback  system 
whose  structure  is  similar 
to  its  own,  making  major 
modifications  in  the 
adopted  system  to  make  it 
more  effective. 

-or- 

Organization  forms  a  task 
force  to  study  the  need 
for  a  feedback  system. 

Task  force  creates  a 
unique  feedback  system 
which  is  custom-tailored 
to  the  needs  of  the 
organization. 
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Table  7 


STRUCTURAL  BARRIERS 


Low 

Medium 

High 

No  feedback  system 
exists.  Leadership  is 
ignorant  of  or  apathetic 
towards  the  need  for  a 
feedback  system. 

Feedback  system  exists, 
but  procedures  used  in  the 
system  are  a  hinderance 
to  the  effectiveness  of  its 
administration. 

Feedback  system  has  few 
procedural  flaws. 
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IV.  Comparison  of  the  Ideal  Feedback  Model  and  the  Air  Force  System 


Introduction. 

Now  that  the  most  important  parameters  of  the  ideal  feedback  system  have 
been  explored,  a  comparison  between  that  oKxiel  and  the  Air  Force  System  will  made. 
Relationships,  or  the  absence  thereof,  between  the  following  parameters  will  be 
explored:  demographic  dissimilarities  of  recipients,  hfequency  of  feedback, 
management  support  of  program,  appropriateness  of  system  with  respect  to  the 
organization,  and  structural  barriers. 

Demographic  Dissimilarities  of  Recipients. 

In  the  Air  Force,  many  of  the  recipients  of  feedback  in  the  enlisted  force  have 
a  common  background.  For  example,  most  enlisted  personnel  have  the  same  level  of 
education  (a  high  school  diploma)  (AF  Magazine,  1992:27).  Almost  all  recipients  of 
feedback  receive  the  same  vocational  education  at  an  Air  Force  technical  school.  On 
the  other  hand,  enlisted  people  have  a  varied  background  when  it  comes  to  salary, 
tenure,  and  age  (AF  Magazine,  1992:26).  This  is  often  true  of  people  who  perform 
tasks  that  require  the  same  skill.  For  example,  a  nineteen-year-old  airman  first  class 
who  is  a  jet  engine  mechanic  is  required  to  repair  engines.  On  the  other  hand,  a 
twenty-ftve-year-old  staff  sergeant  who  is  a  jet  engine  mechanic  probably  spends  a 
substantial  amount  of  his  or  her  time  repairing  jet  engines  (with  some  cursory 
supervisory  duties).  In  essence,  they  peiform  the  same  job,  but  dtere  is  a  significant 
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difference  between  their  pay,  tenure,  and  age.  Accordingly,  the  Air  Force  obtains  a 
medium  ranking  on  this  factor. 

In  the  case  in  which  the  types  and  varieties  of  jobs  are  considered,  the  Air 
Force  also  obtains  a  medium  rank.  Many  of  the  jobs  are  identical,  but  they  require  a 
great  deal  of  specialization. 

Frequency  of  Feedback. 

The  Air  Force  receives  the  highest  rating  with  respect  to  the  frequency  of 
feedback.  A  formal  structure  exists  in  which  the  recipients  are  urged  to  request 
feedback  to  clear  up  any  misunderstandings  about  their  responsibilities  or  their  job 
performance.  In  addition,  supervisors  are  encouraged  to  use  the  feedback  system  to 
counsel  subordinates  on  their  subordinate’s  area  of  responsibility  and  efficiency  of 
performing  tasks.  Added  to  these  unscheduled  feedback  sessions,  are  a  host  of 
required  meetings  which  occur  periodically  throughout  each  assignment  an  enlisted 
person  undergoes.  For  example,  as  mentioned  previously,  30  days  after  starting  a  new 
job  or  being  assigned  a  new  rater  and  180  days  after  this  initial  period  an  airman  or 
NCO  receives  an  informal  feedback  session  (Dept  USAF,  1990:167). 

Management  Support  of  Program. 

The  Air  Force  also  receives  the  highest  mark  in  this  area  as  well.  A  special 
task  force  was  formed  in  1988  to  produce  and  execute  the  most  effective  feedback 
system  possible  (Dept  USAF,  1990:167).  Top  management  was  involved  in  every  step 
of  the  procedure,  during  its  development  and  implementation.  Before  the  new  system 


41 


was  fully  implemented,  a  myriad  of  seminars  and  briefings  were  held  at  the  widest 
possible  dissemination  explaining  the  new  system  to  both  supervisors  and  subordinates 
alike.  After  the  new  system  was  fully  implemented,  leadership  from  the  wing  level  to 
the  highest  echelons  in  the  Air  Force  closely  monitored  its  progress.  Some  minor 
changes  were  made  to  the  system,  and  the  speed  with  which  upper-level  management 
reacted  to  these  minor  glitches  in  the  system  demonstrates  their  keen  interest  in 
ensuring  its  success. 

Appropriateness  of  System  With  Respect  to  the  Organization. 

The  Air  Force  formed  a  special  task  force  to  create  the  ideal  feedback  system 
for  their  organization.  The  feedback  system  they  created  was  custom-tailored  to  meet 
the  special  qualities  of  the  Air  Force  organizational  structure. 

Structural  Barriers. 

The  task  force  the  Air  Force  created  to  study  the  enlisted  evaluation  system 
took  great  pains  to  ensure  that  the  new  feedback  system  would  eliminate  the  structiual 
barriers  which  existed  in  the  old  system.  One  of  the  most  important  barriers  was 
eliminated  by  the  creation  of  an  informal  feedback  system,  which  did  not  exist  in  the 
old  system  (Dept  USAF,  1990:166).  This  informal  feedback  system  allows  the  rater  to 
provide  feedback  to  subordinates  in  a  confidential  way.  By  regulation,  the  rater  and 
the  ratee  are  the  only  ones  who  are  allowed  to  see  the  feedback  worksheet.  As  a 
result,  many  problems  associated  with  formal  feedback  sessions,  such  as  inflated 
ratings,  are  avoided. 
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Another  barrier  that  was  removed  when  the  new  system  was  implemented  was 
that  of  the  infrequency  of  feedback.  Under  the  old  system,  supervisors  were  required 
to  provide  formal  feedback  to  their  subordinate’s  once  a  year  (except  under  special 
circumstances  -  when  an  airman  or  NCX)  changed  raters,  for  example),  and  there  was 
no  system  in  place  to  provide  informal  feedback.  In  the  new  system,  formal  feedback 
is  still  usually  only  provided  once  a  year.  However,  an  informal  feedback  structure 
has  been  instituted  which  establishes  specific  guidelines  as  to  the  minimum  number  of 
times  an  airman  or  NCO  will  receive  feedback.  The  minimum  number  of  mandatory 
informal  feedback  sessions  for  an  airman  arriving  at  a  new  duty  station  is  three  within 
the  first  year  (Dept  USAF,  1990:166).  Hence,  the  number  of  feedback  sessions 
required  by  regulation  has  expanded  from  one  (one  formal  session)  under  the  old 
system  to  four  or  more  (one  formal  session  and  three  or  more  informal  sessions) 
within  the  first  year  of  an  airman  arriving  at  a  new  duty  station. 

Despite  the  frequency  that  an  airman  receives  feedback,  enlisted  personnel 
above  the  rank  of  master  sergeant  do  not  have  to  engage  in  informal  feedback  sessions 
with  their  supervisors  (Dept  USAF,  1990:166).  However,  that  is  not  vo  say  that  they 
cannot  receive  feedback.  If  an  enlisted  person  in  or  above  the  rank  of  master  sergeant 
desires  feedback,  they  can  request  it.  The  same  is  true  of  their  supervisor.  If  the 
supervisor  of  an  individual  who  is  in  the  grade  of  master  sergeant  or  higher  feels  that 
they  need  feedback,  he  or  she  may  give  this  individual  informal  feedback. 
Nevertheless,  under  the  new  system,  there  are  no  institutional  requirements  which 
require  supervisors  to  perform  informal  feedback  with  people  in  the  upper-echelon  of 
the  enlisted  tier.  This  may  be  the  sole  criticism  of  the  evaluation  system.  According 
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to  the  literature  we  studied,  all  people  desire  feedback,  even  those  in  the  upper- 
management  positions  (Longenecker  and  Gioia,  1988:42).  If  this  viewpoint  is 
accurate,  perhaps  the  feedback  system  should  have  required  informal  feedback  sessions 
for  higher  ranking  enlisted  personnel  as  well  as  for  lower  ranking  personnel.  In  any 
case,  the  task  force  was  extremely  successful  in  their  attempt  to  remove  the  structural 
barriers  which  existed  in  the  old  system.  For  that  reason,  the  Air  Force  earns  a  top 
rating  in  this  category. 

When  taking  all  parameters  into  account,  the  Air  Force  feedback  system 
receives  an  overall  excellent  rating.  Some  of  the  variables  which  the  Air  Force  has  to 
contend  with,  such  as  the  demographic  dissimilarities  of  subordinates  receiving 
feedback,  cannot  be  easily  changed,  if  at  all.  In  addition,  the  task  force  did  a  good  job 
of  overcoming  many  of  the  problems  that  were  inherent  in  the  old  system  by 
establishing  an  informal  feedback  system. 

A  diagram  of  the  ideal  feedback  model  is  presented  which  demonstrates  the 
researchers’  overall  evaluation  of  the  Air  Force  feedback  system.  It  is  followed  by 
several  figures  which  list  the  various  characteristics,  broken  down  by  trait,  of  the  Air 
Force  model  with  a  ranking  of  low,  medium,  or  high. 
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High 


Figure  2.  Air  Force  System  Comparison  to  the  Ideal  Model 
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Table  8 


UNITED  STATES  AIR  FORCE 
DEMOGRAPHIC  DISSIMILARITIES  OF  RECIPIENTS 
(BACKGROUND) 


_ Low _ 

There  is  complete  or  near 
complete  diversification 
of  each  recipient’s 
background  with  respect 
to: 

1.  Formal  education 

2.  Technical  education 

3.  Salary 

4.  Tenure 

5.  Age 


Are  two  io  tiiree 
ureas  Hwt  aimstar  in 
eadi  red|lieot’'s 


1.  F<ninitl  etfucatiim 

2.  Teebnicai  edhicaUwi 

4.  .Tnnnre. :  • 

'A|e  ' 


_ High _ 

There  are  at  least  four 
areas  that  are  similar  in 
each  recipient’s 
background  with  respect 
to: 

1.  Formal  education 

2.  Technical  education 

3.  Salary 

4.  Tenure 

5.  Age 


Table  9 


UNITED  STATES  AIR  FORCE 
DEMOGRAPHIC  DISSIMILARITIES  OF  RECIPIENTS 

(JOB) 


Low 

Mtedtiun 

High 

There  are  a  great  number 
of  different  tasks 
requiring  a  high  degree  of 
specialization. 

varielytif 
jobs,  not 

require  a  deal  of 

sp^aiiiad^. 

-or- 

Maiij  of  ^  Jtiribs  are 
IdaitioBl,  but  ihejr 
require  a$  ireai  deai  of 

. 

All  or  most  jobs  are  the 
same  or  require  the  same 
degree  of  specialization. 
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Table  10 


UNITED  STATES  AIR  FORCE 
FREQUENCY  OF  FEEDBACK 


Low 

Medium 

i— I 

Zero.  The  organization 

More  than  once  a  year. 

Feedbaidcis 

has  no  feedback  system. 

However,  feedback  is 

performed  at  tlie 

performed  only  at 

request  of  8ie  redpieiit 

-or- 

specified  intervals. 

or  by  befog  legated  by 
i  foe  mdKHAate^s 

Once  a  year.  Feedback  is 

«%q)ervism’»  as  well  m  at 

performed  as  an 
administration  function 
without  regard  to 
increasing  the 
effectiveness  of  the 

sfi^tled  fotervids. 

organization. 
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Table  11 


UNITED  STATES  AIR  FORCE 
MANAGEMENT  SUPPORT  OF  PROGRAM 


Low 

Medium 

Program  does  not  exist. 
Hence,  leadership  gives 
no  emphasis  to  a  feedback 
system,  since  it  has  not 
bothered  to  implement 
one. 

-or- 

System  exists  in  an 
organization,  but 
administration  of  program 
is  not  enforced  by  the 
leadership  within  the 
organization. 

Program  receives  some 
support  from  upper-level 
management.  Certain 
areas  on  a  performance 
repon  may  be  valid 
because  the  leadership 
within  the  organization 
stresses  those  particular 
domains. 

suppi>rl  fktwi  tettMIup. 
Top  inaiHiganeiil  tidtes 

OB  ocHvo  nile  in 
onsiiring  Hiat  ^ir 
feedback  program  b 
being  imidemented  in 
acc(»rdmice  with 
organiaational  goals. 
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Table  12 


UNITED  STATES  AIR  FORCE 

APPROPRIATENESS  OF  SYSTEM  WITH  RESPECT  TO  ORGANIZATION 


Low 

Organization  adopts  an 
existing  feedback  system 
from  another  organization 
which  has  a  radically 
different  structure  ftom 
their  own.  In  addition, 
the  organization  makes  no 
changes  in  the  adopted 
system  to  make  it  a  more 
effective  instrument. 


Medium 

Organization  adopts  an 
existing  feedback  system 
whose  structure  is  similar 
to  its  own,  making  some 
minor  changes  in  the 
adopted  system  in  an 
attempt  to  make  it  more 
effective. 


fle«diiadc 

IS 

iimilttr  tis 

lit  the 

mSxuptitd  system  to  maite 
It  more 

Organlmtion  forms  a 
tfBk  force  to  study  the 
need  for  a  feedbadt 
system.  Task  fmte 
fsreidm  a  atotue 
^«dhack  system  sdilch 
isciiel(!«ii4aao^ 
itee<ls<dthe 
oitfuiizallon. 
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Table  13 


UNITED  STATES  AIR  FORCE 
STRUCTURAL  BARRIERS 


Low 

Medium 

High 

No  feedback  system 
exists.  Leadership  is 
ignorant  of  or  apaiiietic 
towards  the  need  for  a 
feedback  system. 

Feedback  system  exists, 
but  procedures  used  in  the 
system  are  a  hinderance 
to  the  effectiveness  of  its 
administration. 
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V.  Conclusions  and  Recommendations 


Conclusions. 

There  is  substantial  evidence  to  indicate  that  the  new  Air  Force  feedback 
system  is  an  improvement  over  the  old  design.  Under  the  old  system,  the  only 
regulated  means  of  providing  feedback  was  a  formal  report  which  was  issued,  on  the 
average,  once  a  year.  Since  it  was  a  formal  rating,  it  was  subject  to  several  problems, 
not  the  least  of  which  was  a  tendency  towards  inflationary  ratings.  These  problems 
made  the  system  somewhat  ineffective  with  respect  to  making  it  a  useful  tool  for 
providing  feedback  to  airmen  and  NCOs.  The  new  procedure,  with  its  regulated 
informal  structure,  is  much  more  efficient  at  providing  accurate  and  timely  feedback. 

Recommendations. 

In  theory,  the  Air  Force  has  an  exceptionally  well  crafted  feedback  system. 
Unfortunately,  our  research  was  not  designed  to  test  the  effectiveness  of  its 
administration.  In  other  words,  the  Air  Force  feedback  system  is  well-designed,  but 
the  researchers  were  unable  to  ascertain  if  the  system  is  being  implemented  as 
designed.  This  is  an  important  question  broause  despite  the  etrq)hasis  the  upper-level 
echelon  of  the  Air  Force  places  on  the  program  and  the  solid  theoretical  foundation  of 
the  system,  it  may  be  poorly  implemented. 

In  order  to  test  the  effectiveness  of  administration,  the  researcheia  developed  a 
feedback  questionnaire  which  can  test  this  parameter  by  examining  the  perception  of 
enlisted  personnel’s  views  on  how  well  they  know  their  duties  and  responsibilities. 
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speciHcally  those  connected  with  their  job  (see  Appendix  E).  In  addition,  the 
methodology  behind  the  appropriate  statistical  tests  are  also  included  (see  Appendix 
A).  The  questions  are  placed  in  several  like  categories  for  the  purpose  of  performing 
a  linear  regression  (see  Appendix  B  and  Appendix  D).  The  SAS  program  was 
developed  to  help  the  researcher(s)  test  the  data  obtained  from  the  questionnaire.  The 
researcher(s)  can  evaluate  the  data  obtained  from  the  SAS  output  through  the  use  of 
different  equations  (see  Appendix  C). 

A  multiple  regression  model  was  created  using  nineteen  independent  variables, 
consisting  of  performance  feedback  and  demographic  factors,  and  a  dependent 
variable,  job  knowledge.  The  first  model  that  should  be  developed  would  be  that  of 
the  second  order,  or  complete,  variety.  After  that  model  is  constructed,  a  stepwise 
regression  should  be  performed  to  determine  the  independent  variables  which  add  to 
the  meaningful  predictability  of  the  model  with  respect  to  job  knowledge.  Rom  the 
results  of  the  stepwise  regression,  a  second  truxlel  should  be  developed,  incorporating 
only  the  independent  variables  which  add  significance  to  the  model.  ANOVA  and 
Bonferroni  tests  should  be  conducted  in  an  effort  to  detect  any  biases  that  may  enter 
into  a  study  as  a  result  of  the  questionnaire  or  because  of  the  differences  in  rank 
structure  or  commands. 

Any  persons  attempting  to  use  SAS  language  in  Appendix  E  should  take 
special  precautions  with  respect  to  the  minimum  number  of  cases  that  are  needed  to 
run  the  noodel.  Since  there  are  a  large  number  of  terms,  an  absolute  minimum  of  210 
cases  must  be  used  in  order  to  avoid  receiving  an  ertOT  message.  If  less  than  210 
cases  are  used,  the  SAS  System  will  return  a  message  which  states  the  nxxlel  is  not  of 
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full  rank  (SAS  Institute,  1985:4).  This  simply  means  that  there  are  more  variables 
than  are  data  to  plot  in  the  regression.  If  the  reader  wants  to  perform  a  regression  but 
cannot  obtain  enough  cases  to  run  the  complex  model,  he  or  she  can  use  the  more 
simplified  nxxlel  which  contains  no  interaction  terms. 

It  is  the  researchers  sincerest  desire  that  this  thesis  will  lay  the  groundwork  for 
a  more  in-depth  study  in  which  the  effectiveness  of  the  administration  of  the  enlisted 
evaluation  system  will  be  measured.  Even  if  future  researchers  do  not  use  the  survey 
questionnaire  and  programming  language  we  have  provided  in  this  analysis,  we  hope  it 
will  spark  an  equally  rewarding  research  topic  which  will  yield  valuable  results  to  the 
Air  Force. 
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Appendix  A:  Suggested  Methodology 


General  Procedures. 

The  literature  strongly  suggests  that  a  multiple  regression  analysis  would  be  an 
appropriate  procedure  for  determining  if  there  is  a  relationship  between  performance 
feedback  and  job  knowledge. 

In  order  to  have  an  adequately  predictive  model,  it  is  often  necessary  to  have 
several  independent  variables  in  a  regression  analysis  (McClave  and  Benson, 
1991:522).  In  our  study,  the  dependent  variable,  job  knowledge,  could  have  been  a 
function  of  the  independent  variables,  which  could  have  consisted  of  demographic 
parameters  and  performance  appraisal  effectiveness  issues.  Table  14  provides  a  list  of 
the  independent  variables  that  could  be  used: 

Table  14 

INDEPENDENT  VARIABLES 


INDEPENDENT 

VARIABLE 


Xj 

*3 

*5 


x. 


DESCRIPTION 

Rank 
Skill  level 

Total  number  of  years  on  active  duty 

Highest  PME 

Most  recent  PME 

PME  completed  by  in-residence  or 
correspon^nce 

Supervisor  (yes  or  no) 
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Table  14  (Continued) 


INDEPENDENT  VARIABLES 


INDEPENDENT 

VARIABLE 


X, 

XlO 

Xn 

Xi2 

X,3 

X,4 

XlS 

Xj7 

Xl8 

X,9 


DESCRIPTION 


Most  recent  Feedback  performed 

Performance  feedback  caused  fear, 
anxiety 

Performance  feedback  was  a  fair 
evaluation 

Performance  feedback  was  accurate 

Performance  feedback  was 
understandable 

Performance  feedback  was  encouraging 

Pwfoimance  feedback  showed  areas 
which  needed  improvement 

Performance  feedback  helped  me 
understand  my  job  better 

Performance  feedback  was  poorly  timed 

Performance  feedback  was  frustrating 

Performance  feedback  was  disorganized 

Performance  feedback  was  conducted  by 
a  supervisor  who  was  not  well  trained 


The  first  seven  independent  variables,  x,  through  are  factors  which  affect 
job  knowledge  but  are  not  related  to  performance  feedback.  The  last  twelve 
independent  variables,  x,  through  x,,,  ate  related  to  performance  feedback.  From  the 
model  that  could  be  developed  from  this  research,  the  relationship  between 
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performance  feedback  and  job  knowledge  could  be  examined.  Ii  addition,  a  study 
could  be  undertaken  with  the  hope  of  discovering  the  most  important  performance 
feedback  variables  with  respect  to  predicting  job  knowledge. 

The  method  of  least  squares  is  the  procedure  used  to  fit  a  multiple  regression 
model.  The  least  squares  method  is  a  process  which  builds  a  model  which  minimizes 
the  sum  of  squared  errors  (SSE)  (McClave  and  Benson,  1991:523).  A  model  could  be 
developed  with  the  assistance  of  the  SAS  System,  a  computer  program  in  statistical 
applications  (Schlotzhauer  and  Littell,  1987:5). 

Stepwise  Regression.  A  second-order  model  could  be  created  from  the  original 
nineteen  independent  variable*'  measured  in  this  research.  The  reason  a  second-order 
version  should  be  chosen  over  a  first-order  model  is  that  it  is  possible  that  a  second- 
order  model  may  contribute  more  information  to  the  predictability  of  the  dependent 
variable  than  the  first-order  type.  In  the  event  that  a  first-order  model  does  actually 
contribute  more  than  that  of  the  second-order  design,  the  process  of  completing  the 
stepwise  regression  would  reduce  the  model  from  a  second-order  model  to  a  first-order 
model. 

It  is  important  to  observe  that  by  adding  the  original  nineteen  variables  to  the 
interaction  terms  and  second-order  terms,  the  original  model  would  contain  210 
independent  variables.  Obviously,  some  of  these  terms  would  have  more  predictive 
value  than  others.  Hence,  it  would  be  necessary  to  determine  how  much  each  of  these 
factors  would  contribute  to  the  usefulness  of  the  model.  For  example,  it  could 
surmised  that  it  might  be  possible  that  the  accuracy  of  performance  feedback  would 
affect  job  knowledge  more  than  the  fairness  of  performance  feedback.  In  order  to 
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determine  how  the  variables  weighed  against  job  knowledge,  a  technique  called 
stepwise  regression  could  be  employed  since  it  is  one  of  the  most  precise  methods 
available  to  construct  a  model  with  numerous  independent  variables  (McClave  and 
Benson,  1991:671).  This  technique  is  especially  useful  when  the  model  contains  a 
large  number  of  multivariate  interactions,  as  would  the  model  in  this  research 
(McClave  and  Benson,  1991:671).  Again,  this  process  could  be  accomplished  with  the 
aid  of  a  computer  using  the  SAS  System.  In  the  SAS  System,  the  computer  first 
identifies  the  dependent  variable  and  independent  variables.  The  next  part  of  the 
procedure,  which  is  called  step  1,  consists  of  the  program  fitting  all  possible  one- 
variable  versions  of  the  model  to  the  data.  The  computer  selects  the  variable  of  the 
model  that  produced  the  largest  t  value.  In  other  words,  the  computer  picks  the  one- 
variable  model  which  was  the  best  predictor  of  the  dependent  variable,  job  knowledge. 
This  variable  is  named  P,x,  (McClave  and  Benson,  1991:671).  In  the  second  step,  the 
program  checks  the  remaining  variables  for  the  best  two-variable  form  of  the  model. 
SAS  accomplishes  this  task  by  checking  all  the  remaining  variables,  one  at  a  time, 
with  the  variable  that  was  chosen  in  step  1.  After  the  new  variable,  named  PjXj,  is 
added,  the  t  value  is  examined  again  to  ensure  that  it  has  remained  significant,  at  the 
specified  a  level.  If  the  r  value  is  no  longer  significant,  the  PjX,  variable  is  removed, 
and  the  program  explores  the  remaining  independent  variables  that  wiU  combine  with 
pjXj  to  yield  the  most  signiricant  t  value.  The  SAS  System  examines  the  t  value  from 
step  1  to  step  2  because  the  variables  Pi  and  p2  may  be  correlated  in  some  way.  If 
they  are  correlated,  the  significance  of  the  t  value  will  change.  The  program  continues 
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to  add  variables  to  the  model  until  there  are  no  variables  left  which  significantly 
contribute  to  the  model’s  predictive  value,  at  a  given  a  level  (K  ?lave  and  Benson, 
1991:672). 

It  should  be  pointed  out  that  although  this  technique  is  useful  in  that  this 
process  eliminates  variables  which  do  not  significantly  add  to  the  usefulness  of  the 
model,  the  model  may  be  flawed.  It  is  possible  that  an  important  independent 
variable,  unknown  to  the  researchers,  may  be  omitted  from  the  variables  that  were 
included  in  the  stepwise  regression.  If  such  a  variable  is  excluded,  this  model’s 
predictive  value  may  not  be  as  useful  as  this  procedure  might  otherwise  indicate 
(McClave  and  Benson,  1991:672). 

Adjusted  Value.  After  the  final  model  is  developed,  its  usefulness  could  be 
examined  by  calculating  its  R*  value.  This  value  ranges  from  0  to  1.  A  value  of  0 
implies  that  there  is  absolutely  no  fit  between  the  model  and  the  data.  On  the  other 
hand,  a  value  of  1  indicates  a  perfect  correlation  between  the  data  and  the  model.  In 
this  latter  case,  every  point  from  the  sample  passes  through  the  model.  The  higher  the 
value,  the  more  useful  the  model  (McClave  and  Benson,  1991:541).  However, 
since  the  number  of  independent  variables  in  this  study  is  large,  an  adjusted  R^  value 
should  be  used.  Hie  difference  between  an  adjusted  and  unadjusted  value  is  that  the 
unadjusted  reading  increases  as  the  number  of  independent  variables  increase,  although 
there  may  not  be  a  true  relationship  between  the  independent  variables  and  dependent 
variable  (Statistix,  1991:178).  An  adjusted  R^  value  uses  a  formula  to  take  into 
account  this  phenomenon  and  adjusts  the  value  according  to  the  number  of 
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independent  variables.  It  should  be  pointed  out  that  there  is  sometimes  a  danger  of 
observing  a  negative  value  when  it  is  adjusted  for  variable  size  (Statistix, 

1991:178). 

Instrument  Development  and  Testing. 

There  were  several  variables  which  lent  themselves  to  analysis.  In  this 
particular  study,  we  developed  several  pertinent  questions  relating  to  the  dependent 
variable,  job  knowledge,  which  were  relevant  to  airmen  and  NCOs.  For  questions 
dealing  with  the  independent  variables,  performance  appraisal  effectiveness  issues,  this 
study  used  an  appraisal  interview  questionnaire  developed  by  John  M.  Ivancevich  and 
the  GAN  AT  Company  (Gibson  and  others,  1991:700).  With  some  slight  modifications 
to  fit  this  particular  analysis,  the  questionnaire  remained  faithful  to  the  original 
research  instrument  developed  by  GANAT  (Emory,  1991:374).  Also,  several 
demographic  questions  were  posed  to  each  respondent  that  directly  related  to  their  job 
knowledge. 

Description  of  Population  and  Sample. 

We  suggest  that  the  sample  consist  of  at  least  2,680  enlisted  personnel  out  of 
population  of  200,(X)0,  drawn  by  simple  random  sample.  The  population  could  contain 
all  of  the  enlisted  members  of  Air  Training  Command,  Air  Force  Mobility  Command, 
Air  Ckimbat  Command,  for  example. 
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Data  Collection  Plan. 


As  is  consistent  with  good  research,  the  survey  instrument,  a  questionnaire, 
begins  with  simple  items  and  moves  on  to  more  complex  issues  (Emory,  1991:370). 
Accordingly,  questions  001  through  009  deal  strictly  with  demographic  data. 

The  questions  that  deal  with  population  characteristics  are  relatively 
straightforward.  However,  the  questions  dealing  with  the  independent  variables 
concerning  performance  feedback  effectiveness  required  a  more  intricate  construction. 
According  to  the  literature,  it  is  prudent  to  disguise  the  purpose  of  the  survey 
instrument  in  an  attempt  to  prevent  any  significant  biases  from  entering  the  study 
(Emory,  1991:352).  One  of  the  ways  this  study  reduced  the  chance  of  bias  was  to 
assemble  the  questionnaire  in  such  a  way  as  to  ask  each  question  (excluding  questions 
requesting  demographic  information)  concerning  a  particular  area  of  interest  more  than 
once.  Each  of  the  questions  in  each  area  requested  the  same  information  but  were 
phrased  in  a  different  manner  and  were  randomly  distributed  in  the  questionnaire. 

Some  of  the  questions  were  also  worded  in  such  a  way  as  to  illicit  a  reverse  response 
when  compared  to  the  other  questions.  The  use  of  this  technique  allowed  the  authors 
of  this  study  to  check  for  consistency  between  each  of  the  questions  that  were  grouped 
in  similar  areas.  If  no  bias  had  been  introduced  into  the  questionnaire,  one  would 
expect  the  responses  for  each  question,  within  a  given  area,  should  be  the  same. 

ANOVA  Tables.  One  of  the  simplest  ways  to  check  for  the  possibility  of  bias 
in  the  survey  instrument  was  to  perform  a  test  of  a  completely  randtnnized  design  by 
comparing  treatment  means  between  the  questions  grouped  in  like  dtnnains  (McClave 
and  Benson,  1991:866).  The  mean  calculated  from  a  given  question  in  a  given  area 
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was  compared  against  the  means  of  the  other  questions  on  the  same  subject  by  using 
an  ANOVA  table.  If  the  survey  instrument  were  unbiased,  the  data  should  indicate 
that  there  was  no  significant  difference  betvseen  the  treatment  means  of  questions  in 
like  areas.  Also  the  treatment  mean  which  dealt  with  job  knowledge  could  be 
examined  between  the  three  enlisted  rank  tiers  and  between  commands.  This  task 
should  be  accomplished  in  order  to  investigate  the  possibility  that  there  may  be  a 
significant  variation  between  rank  structures  or  agencies  with  respect  to  job 
knowledge. 

Bonferroni  Procedure.  The  ANOVA  test  reveals  whether  there  is  a  significant 
difference  between  two  or  more  treatment  means.  However,  the  results  do  not  indicate 
which  treatment  means  differ  (in  most  cases,  there  were  more  than  two  treatment 
means  being  examined  in  each  area  of  interest  in  this  study)  or  by  how  much  they 
vary.  If  a  significant  variation  was  found  between  at  least  two  treatment  means  in  a 
given  area,  the  Bonferroni  procedure  should  be  applied  to  find  how  means  differed  and 
to  what  extent  they  vary.  This  procedure  should  be  chosen  over  others  because  it  is 
easy  to  implement,  and  the  results  observed  from  using  this  method  are  conservative 
in  nature  (McClave  and  Benson,  1991:873). 

Summary. 

A  multiple  regression  model  could  be  created  using  die  nineteen  independent 
variables,  performance  feedback  and  demographic  factors,  against  the  dependent 
variable,  job  knowledge.  The  first  model  developed  should  be  of  the  second  order,  or 
complete,  variety.  After  that  model  is  constructed,  a  stepwise  regression  should  be 
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poformed  to  determine  the  independent  variables  which  add  to  the  meaningful 
predictability  of  the  model  with  respect  to  job  knowledge.  From  the  results  of  the 
stepwise  regression,  a  second  model  should  be  developed,  incorporating  only  the 
independent  variables  which  add  significance  to  die  model.  ANOVA  and  Bonfeironi 
tests  should  be  conducted  in  an  effort  to  detect  any  biases  that  may  have  entered  into 
the  study  as  a  result  of  the  questionnaire  or  because  of  the  differences  in  rank  structure 
or  commands. 
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Appendix  B:  Job  Knowledge  Question  Groupings 


The  following  information  lists  each  area  that  was  covered  concerning  the 
dependent  variable  and  independent  variables.  After  a  brief  description  of  each  area, 
all  of  the  questions  that  were  asked  in  that  particular  area  (as  they  actually  appeared  in 
the  questionnaire)  are  given: 


Question  Set  1.  (NCOs  and  airmen  understand  responsibilities) 

010.  I  understand  my  responsibilities  as  an  airman/NCO. 

012.  My  obligations  as  an  airman/NCO  are  clear  to  me. 

020.  I  do  not  understand  my  duties  as  an  airman/NCO. 

(A  reverse  response,  in  relation  to  the  other  questions  in  this  area,  is  expected.) 

(Question  Set  2.  (NCOs  and  airmen  skill  levels  reflect  true  level  of  job  knowledge) 

Oil.  I  feel  my  current  skill  level  is  commensurate  with  my  capacity  to  accomplish  my 
job. 

014.  My  skill  level  does  not  do  a  good  job  of  measuring  my  ability  to  do  my  job.  (A 
reverse  response,  in  relation  to  the  other  questions  in  this  area,  is  expected.) 

016.  My  job  proficiency  is  accurately  reflected  by  my  skill  level. 


(^estion  Set  3.  (NCOs  and  airmen  understand  mission  of  their  squadron) 

015.  1  understand  the  importance  of  my  position  within  the  squadron. 

019.  I  realize  the  importance  of  my  job  v/ith  respect  to  accomplishing  the  goals  of  my 
squadron. 

021.  The  way  in  which  my  work  impacts  the  effectiveness  of  the  squadron  is  not 
clear  to  me.  (A  reverse  response,  in  relation  to  the  other  questions  in  this  area,  is 
expected.) 
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Question  Set  4  (NCX)s  and  airmen  understand  mission  of  their  wing) 

013.  I  have  a  clear  understanding  of  the  role  I  play  in  accomplishing  the  wing’s 
mission. 

017.  I  understand  the  inqportance  of  my  position  within  the  wing. 

018.  The  way  in  which  my  work  impacts  the  effectiveness  of  the  wing  is  not  clear  to 
me.  (A  reverse  response  in  relation  to  the  other  questions  in  this  area,  is  expected.) 

Performance  Feedback  Question  Groupings. 

Question  Set  1  (performance  feedback  caused  fear,  anxiety) 

027.  The  feedback  session  really  raised  my  anxiety  level. 

031.  I  dreaded  the  actual  feedback  session  itself. 

Question  Set  2  (perfonnance  feedback  was  a  fair  evaluation) 

023.  The  discussion  of  my  performance  during  the  feedback  session  was  covered 
objectively. 

026.  The  feedback  session  was  fair  in  every  respect. 

032.  The  supervisor  was  straightforward  in  all  phases  of  the  feedback  session. 

043.  During  the  feedback  session,  my  performance  was  analyzed  fairly. 

046.  I  was  worried  that  the  feedback  session  worksheet  would  not  be  kept 
confldential. 

(^esdon  Set  3  (performance  feedback  was  accurate) 

022.  The  performance  feedback  session  coveted  my  entire  job. 

024.  The  feedback  session  was  accurately  conducted. 

044.  I  was  often  upset  because  the  feedback  data  were  inaccurate.  (A  reverse 
response,  in  relation  to  the  other  questions  in  this  area,  is  expected.) 
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04S.  My  record,  as  it  was  introduced  in  the  feedback  session,  contained  no  errors. 
047.  Specific  examples  of  my  performance  were  cited. 

Question  Set  4  (performance  feedback  was  understandable) 

025.  I  didn’t  have  to  ask  for  clarification.  (A  reverse  response,  in  relation  to  the 
other  questions  in  this  area,  is  expected.) 

028.  The  purpose  of  the  feedback  session  was  not  clear  to  nte. 

037.  I  disliked  the  feedback  session  because  the  intent  was  not  clear. 

Question  Set  5  (performance  feedback  was  encouraging) 

030.  The  feedback  session  was  encouraging  m  me  personally. 

✓ 

033.  The  feedback  session  gave  me  some  direction  and  purpose. 

Question  Set  6  (performance  feedback  showed  areas  which  needed  improvement) 
034.  The  feedback  session  really  pinpointed  areas  for  improvement. 

039.  The  feedback  session  has  been  my  guide  for  correcting  weaknesses. 

(^estion  Set  7  (performance  feedback  helped  me  understand  my  job  better) 

029.  The  feedback  session  really  made  me  think  about  working  smarter  on  the  job. 

040.  I  understood  the  meaning  of  each  performance  area  better  after  the  feedback 
session. 

(^estion  Set  8  (performance  feedback  was  poorly  timed) 

041.  The  feedback  session  was  too  rushed. 

042.  I  received  no  advance  notice  about  the  feedback  session. 
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Question  Set  9  (performance  feedback  was  frustrating) 

035.  The  feedback  session  was  frustrating. 

Question  Set  10  (peiformance  feedback  was  disorganized) 

036.  The  feedback  session  was  disorganized. 

Question  Set  11  (peiformance  feedback  was  conducted  by  a  supervisor  who  was  not 
well  trained) 

038.  The  supervisor  who  conducted  the  feedback  session  was  not  well  trained. 


67 


Appendix  C:  Formulas 
Sum  of  Squares  for  Treatment  (SST) 

E  ( 5^  ■  y 


Sum  of  Squares  for  Error  (SSE) 

E  ( -  yi )" 


Mean  Square  for  Treatments  (MST) 

SST 
P  -  1 

Mean  Square  for  Error  (MSE) 

SSE 
n  -  p 

F  Statistic 

F  = 

'  MSE 

Regression  Residual 

y-S 


Standard  Deviation  of  Regression  Residuals 


s  - 


SSE 


y\n-{k*l) 
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Appendix  D:  SAS  Language 


OPTIONS  PAGESIZE=66  LINESIZE=70  NODATE; 

FILENAME  DATASET  ’PRETEST.DAT’; 

DATA  PRETEST; 

INFILE  PRETEST; 

INPUT  TESTNO  1-8  Q1  9  Q2  10  Q3  11  Q4  12  Q5  13  Q6  14  Q7  15  Q8  16 
Q9  17  QIO  18  Qll  19  Q12  20  Q13  21  Q14  22  Q15  23  Q16  24  Q17  25  Q18 
26  Q19  27  Q20  28  Q21  29  Q22  30  Q23  31  Q24  32  Q25  33  Q26  34  Q27  35 

Q28  36  Q29  37  Q30  38  Q31  39  Q32  40  Q33  41  Q34  42  Q35  43  Q36  44 

Q37  45  Q38  46  Q39  47  Q40  48  Q41  49  Q42  50  Q43  51  Q44  52  Q45  53 

Q46  54  Q47  55  Q48  56  Q49  57; 

QSl  =  (QIO  +  Q12  +  (8  -  Q20))/3; 

QS2  =  (Qll  +  Q16  +  (8  -  Q14))/3; 

QS3  =  (Q15  +  Q19  +  (8  -  Q21))/3; 

QS4  =  (Q13  +  Q17  +  (8  -  Q18))/3; 

JOBKNO  =  (QSl  +  QS2  +  QS3  +  QS4)/4; 

PFQSl  =Q1; 

PFQS2  =  Q3; 

PFQS3  =  Q4; 

PFQS4  =  Q5; 

PFQS5  =  Q7; 

PFQS6  =  Q6; 

PFQS7  =  Q49; 

PFQS8  =  Q8; 

PFQS9  =  (Q27  +  Q31)/2; 

PFQSIO  =  (Q23  +  Q26  +  Q32  +  Q43  +  Q46)/5; 

PFQSll  =  (Q22  +  Q24  +  (344  +  Q45  +  (347)/5; 

PFQS12  =  (Q25  +  Q28  +  Q37)/3; 

PFQSl  3  =  (Q30  +  Q33)/2; 

PFQS14  =  (Q34  +  Q39)/2; 

PFQS15  =  (Q29  +  <340)/2; 

PFQS16  =  «341  +  <342)/2; 

PFQS17  =  Q35; 

PFQS18  =  Q36; 

PFQS19  =  Q38; 

Q1X2  =  PFQSl  ♦PFQS2; 

Q1X3  =  PFQS1*PFQS3; 

Q1X4  =  PFQSl  *PFQS4; 

Q1X5  =  PFQSl  *PFQS5; 

Q1X6  =  PFQSl  *PFQS6; 

Q1X7  =  PFQSl  *PFQS7; 
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Q1X8  =  PFQS1*PFQS8; 
Q1X9  =  PFQS1*PFQS9; 
QIXIO  =  PFQS1*PFQS10 
QlXll  =PFQS1*PFQS11 
Q1X12  =  PFQS1*PFQS12 
Q1X13  =  PFQS1*PFQS13 
Q1X14  =  PFQS1*PFQS14 
Q1X15  =  PFQS1*PFQS15 
Q1X16  =  PFQS1*PFQS16 
Q1X17  =  PFQS1*PFQS17 
Q1X18  =  PFQS1*PFQS18 
Q1X19  =  PFQS1*PFQS19 

Q2X3  =  PFQS2*PFQS3; 
Q2X4  =  PFQS2*PFQS4; 
Q2X5  =  PFQS2*PFQS5; 
Q2X6  =  PFQS2*PFQS6; 
Q2X7  =  PFQS2*PFQS7; 
Q2X8  =  PFQS2*PFQS8; 
Q2X9  =  PFQS2*PFQS9; 
Q2X10  =  PFQS2*PFQS10 
Q2XI 1  =  PFQS2*PFQS  1 1 
Q2X12  =  PFQS2*PFQS12 
Q2X13  =  PFQS2*PFQS13 
Q2X14  =  PFQS2*PFQS14 
Q2X15  =  PFQS2*PFQS15 
Q2X16  =  PFQS2*PFQS16 
Q2X17  =  PFQS2*PFQS17 
Q2X18  =  PFQS2*PFQS18 
Q2X19  =  PFQS2*PFQS19 

Q3X4  =  PFQS3*PFQS4; 
Q3X5  =  PFQS3*PFQS5; 
Q3X6  =  PFQS3*PFQS6; 
Q3X7  =  PFQS3*PFQS7; 
Q3X8  =  PFQS3*PFQS8; 
Q3X9  =  PFQS3*PFQS9; 
Q3X10  =  PFQS3*PFQS10 
Q3X11  =  PFQS3*PFQS11 
Q3X12  =  PFQS3*PFQS12 
Q3X13  =  PFQS3*PFQSI3 
Q3X14  =  PFQS3*PFQS14 
Q3X15  =  PFQS3*PFQS15 
Q3X16  =  PFQS3*PFQS16 
Q3X17  =  PFQS3*PFQS17 
Q3X18  =  PFQS3*PFQS18 


Q3X19  =  PFQS3*PFQS19; 
Q4X5  =  PFQS4*PFQS5; 
Q4X6  =  PFQS4*PFQS6; 
Q4X7  =  PFQS4*PFQS7; 
Q4X8  =  PFQS4*PFQS8; 
Q4X9  =  PFQS4*PFQS9; 
Q4X10  =  PFQS4*PFQS10; 
Q4X11  =  PFQS4*PFQS11; 
Q4X12  =  PFQS4*PFQS12; 
Q4X13  =  PFQS4*PFQS13; 
Q4X14  =  PFQS4*PFQS14; 
Q4X15  =  PFQS4*PFQS15; 
Q4X16  =  PFQS4*PFQS16; 
Q4X17  =  PFQS4*PFQS17; 
Q4X18  =  PFQS4*PFQS18; 
Q4X19  =  PFQS4*PFQS19; 

Q5X6  =  PFQS5*PFQS6; 
Q5X7  =  PFQS5*PFQS7; 
Q5X8  =  PFQS5*PFQS8; 
Q5X9  =  PFQS5*PFQS9; 
Q5X10  =  PFQS5*PFQS10 
Q5X11  =  PFQS5*PFQS11 
Q5X12  =  PFQS5*PFQS12 
Q5X13  =  PFQS5*PFQS13 
Q5X14  =  PFQS5*PFQS14 
Q5X15  =  PFQS5*PFQS15 
Q5X16  =  PFQS5*PFQS16 
Q5X17  =  PFQS5*PFQS17 
Q5X18  =  PFQS5*PFQS18 
Q5X19  =  PFQS5*PFQS19 

Q6X7  =  PFQS6*PFQS7; 
Q6X8  =  PFQS6*PFQS8; 
Q6X9  =  PFQS6*PFQS9; 
Q6X10  =  PFQS6*PFQS10 
Q6X11  =  PFQS6*PFQS11 
Q6X12  =  PFQS6*PFQS12 
Q6X13  =  PFQS6*PFQS13 
Q6X14  =  PFQS6*PFQS14 
Q6X15  =  PFQS6*PFQS15 
Q6X16  =  PFQS6*PFQS16 
Q6X17  =  PFQS6*PFQS17 
Q6X18  =  PFQS6*PFQS18 
Q6X19  =  PFQS6*PFQS19 
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Q7X8  =  PFQS7*PFQS8; 
Q7X9  =  PFQS7*PFQS9; 
Q7X10  =  PFQS7*PFQS10; 
Q7X11  =  PFQS7*PFQS11; 
Q7X12  =  PFQS7*PFQS12; 
Q7X13  =  PFQS7*PFQS13; 
Q7X14  =  PFQS7*PFQS14; 
Q7X15  =  PFQS7*PFQS15; 
Q7X16  =  PFQS7*PFQS16; 
Q7X17  =  PFQS7*PFQS17; 
Q7X18  =  PFQS7*PFQS18; 
Q7X19  =  PFQS7*PFQS19; 

Q8X9  =  PFQS8*PFQS9; 
Q8X10  =  PFQS8*PFQS10; 
Q8X1 1  =  PFQS8*PFQS  1 1 ; 
Q8X12  =  PFQS8*PFQS12; 
Q8X13  =  PFQS8*PFQS13; 
Q8X14  =  PFQS8*PFQS14; 
Q8X15  =  PFQS8*PFQS15; 
Q8X16  =  PFQS8*PFQS16; 
Q8X17  =  PFQS8*PFQS17; 
Q8X18  =  PFQS8*PFQS18; 
Q8X19  =  PFQS8*PFQS19; 

Q9X10  =  PFQS9*PFQS10; 
Q9X11  =  PFQS9*PFQS11; 
Q9X12  =  PFQS9*PFQS12; 
Q9X13  =  PFQS9*PFQS13; 
Q9X14  =  PFQS9*PFQS14; 
Q9X15  =  PFQS9*PFQS15; 
Q9X16  =  PFQS9*PFQS16; 
Q9X17  =  PFQS9*PFQS17; 
Q9X18  =  PFQS9*PFQS18; 
Q9X19  =  PFQS9*PFQS19; 

QlOXll  =  PFQS10*PFQS11 
Q10X12  =  PFQS10*PFQS12 
Q10X13  =  PFQS10*PFQS13 
Q10X14  =  PFQS10*PFQS14 
Q10X15  =  PFQS10*PFQS15 
Q10X16  =  PFQS10*PFQS16 
Q10X17  =  PFQS10*PFQS17 
Q10X18  =  PFQS10*PFQS18 
Q10X19  =  PFQS10*PFQS19 


Q11X12  =  PFQS11*PFQS12; 
Q11X13  =  PFQS11*PFQS13; 
Q11X14  =  PFQS11*PFQS14; 
Q11X15  =  PFQS11*PFQS15; 
Q11X16  =  PFQS11*PFQS16; 
Q11X17  =  PFQS11*PFQS17; 
Q11X18  =  PFQS11*PFQS18; 
Q11X19  =  PFQS11*PFQS19; 

Q12X13  =  PFQS12*PFQS13; 
Q12X14  =  PFQS12*PFQS14; 
Q12X15  =  PFQS12*PFQS15; 
Q12X16  =  PFQS12*PFQS16; 
Q12X17  =  PFQS12*PFQS17; 
Q12X18  =  PFQS12*PFQS18; 
Q12X19  =  PFQS12*PFQS19; 

Q13X14  =  PFQS13*PFQS14; 
Q13X15  =  PFQS13*PFQS15; 
Q13X16  =  PFQS13*PFQS16; 
Q13X17  =  PFQS13*PFQS17; 
Q13X18  =  PFQS13*PFQS18; 
Q13X19  =  PFQS13*PFQS19; 

Q14X15  =  PFQS14*PFQS15; 
Q14X16  =  PFQS14*PFQS16; 
Q14X17  =  PFQS14*PFQS17; 
Q14X18  =  PFQS14*PFQS18; 
Q14X19  =  PFQS14='PFQS19; 

Q15X16  =  PFQS15*PFQS16; 
Q15X17  =  PFQS15*PFQS17; 
Q15X18  =  PFQS15*PFQS18; 
Q15X19  =  PFQS15*PFQS19; 

Q16X17  =  PFQS16*PFQS17; 
Q16X18  =  PFQS16*PFQS18; 
Q16X19  =  PFQS16*PFQS19; 

Q17X18  =  PFQS17*PFQS18; 
Q17X19  =  PFQS17*PFQS19; 

Q18X19  =  PFQS18*PFQS19; 

SQXl  =  PFQS1*PFQS1; 
SQX2  =  PFQS2*PFQS2; 


SQX3  =  PFQS3*PFQS3; 
SQX4  =  PFQS4*PFQS4; 
SQX5  =  PFQS5*PFQS5; 
SQX6  =  PFQS6*PFQS6; 
SQX7  =  PFQS7*PFQS7; 
SQX8  =  PFQS8*PFQS8; 
SQX9  =  PFQS9*PFQS9; 
SQXIO  =  PFQS10*PFQS10 
SQXll  =  PFQS11*PFQS11 
SQX12  =  PFQS12*PFQS12 
SQX13  =  PFQS13*PFQS13 
SQX14  =  PFQS14*PFQS14 
SQX15  =  PFQS15*PFQS15 
SQX16  =  PFQS16*PFQS16 
SQX17  =  PFQS17*PFQS17 
SQX18  =  PFQS18*PFQS18 
SQX19  =  PFQS19*PFQS19 
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PROC  PRINT  DATA=PRETEST; 

TITLE  ’DATA  SET’; 

PROC  REG  DATA=PRETEST; 

MODEL  JOBKNO=PFQSl  PFQS2  PFQS3  PFQS4  PFQS5  PFQS6  PFQS7 
PFQS8  PFQS9  PFQSIO  PFQSll  PFQS12  PFQS13  PFQS14  PFQS15  PFQS16  PFQS17 
PFQS18  PFQS19  Q1X2  Q1X3  Q1X4  Q1X5  Q1X6  Q1X7  Q1X8  Q1X9  QIXIO  QlXll 
Q1X12  Q1X13  Q1X14  Q1X15  Q1X16  Q1X17  Q1X18  Q1X19  Q2X3  Q2X4  Q2X5 
Q2X6  Q2X7  Q2X8  Q2X9  Q2X10  Q2X11  Q2X12  Q2X13  Q2X14  Q2X15  Q2X16 
Q2X17  Q2X18  Q2X19  Q3X4  Q3X5  Q3X6  Q3X7  Q3X8  Q3X9  Q3X10  Q3X11  Q3X12 
Q3X13  Q3X14  Q3X15  Q3X16  Q3X17  Q3X18  Q3X19  Q4X5  Q4X6  Q4X7  Q4X8 
Q4X9  Q4X10  Q4X11  Q4X12  Q4X13  Q4X14  Q4X15  Q4X16  Q4X17  Q4X18  Q4X19 
Q5X6  Q5X7  Q5X8  Q5X9  Q5X10  Q5X11  Q5X12  Q5X13  Q5X14  Q5X15  Q5X16 
Q5X17  Q5X18  Q5X19  Q6X7  Q6X8  Q6X9  Q6X10  Q6X11  Q6X12  Q6X13  Q6X14 
Q6X15  Q6X16  Q6X17  Q6X18  Q6X19  Q7X8  Q7X9  Q7X10  Q7X11  Q7X12  Q7X13 
Q7X14  Q7X15  Q7X16  Q7X17  Q7X18  Q7X19  Q8X9  Q8X10  Q8X11  Q8X12  Q8X13 
Q8X14  Q8X15  Q8X16  Q8X17  Q8X18  Q8X19  Q9X10  Q9X11  Q9X12  Q9X13  Q9X14 
Q9X15  Q9X16  Q9X17  Q9X18  Q9X19  QlOXll  Q10X12  Q10X13  Q10X14  Q10X15 
Q10X16  Q10X17  Q10X18  Q10X19  Q11X12  Q11X13  Q11X14  Q11X15  Q11X16 
Q11X17  Q11X18  Q11X19  Q12X13  Q12X14  Q12X15  Q12X16  Q12X17  Q12X18 
Q12X19  Q13X14  Q13X15  Q13X16  Q13X17  Q13X18  Q13X19  Q14X15  Q14X16 
Q14X17  Q14X18  Q14X19  Q15X16  Q15X17  Q15X18  Q15X19  Q16X17  Q16X18 
Q16X19  Q17X18  Q17X19  Q18X19  SQXl  SQX2  SQX3  SQX4  SQX5  SQX6  SQX7 
SQX8  SQX9  SQXIO  SQXll  SQX12  SQX13  SQX14  SQX15  SQX16  SQX17  SQX18 
SQX19; 
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PROC  STEPWISE  DATA=PRETEST; 

MODEL  JOBKNO=PFQSl  PFQS2  PFQS3  PFQS4  PFQS5  PFQS6  PFQS7  PFQS8 
PFQS9  PFQSIO  PFQSll  PFQS12  PFQS13  PFQS14  PFQS15  PFQS16  PFQS17 
PFQS18  PFQS19  Q1X2  Q1X3  Q1X4  Q1X5  Q1X6  Q1X7  Q1X8  Q1X9  QIXIO  QlXll 
Q1X12  Q1X13  Q1X14  Q1X15  Q1X16  Q1X17  Q1X18  Q1X19  Q2X3  Q2X4  Q2X5 
Q2X6  Q2X7  Q2X8  Q2X9  Q2X10  Q2X11  Q2X12  Q2X13  Q2X14  Q2X15  Q2X16 
Q2X17  Q2X18  Q2X19  Q3X4  Q3X5  Q3X6  Q3X7  Q3X8  Q3X9  Q3X10  Q3X11  Q3X12 
Q3X13  Q3X14  Q3X15  Q3X16  Q3X17  Q3X18  Q3X19  Q4X5  Q4X6  Q4X7  Q4X8 
Q4X9  Q4X10  Q4X11  Q4X12  Q4X13  Q4X14  Q4X15  Q4X16  Q4X17  Q4X18  Q4X19 
Q5X6  Q5X7  Q5X8  Q5X9  Q5X10  Q5X11  Q5X12  Q5X13  Q5X14  Q5X15  Q5X16 
Q5X17  Q5X18  Q5X19  Q6X7  Q6X8  Q6X9  Q6X10  Q6X11  Q6X12  Q6X13  Q6X14 
Q6X15  Q6X16  Q6X17  Q6X18  Q6X19  Q7X8  Q7X9  Q7X10  Q7X11  Q7X12  Q7X13 
Q7X14  Q7X15  Q7X16  Q7X17  Q7X18  Q7X19  Q8X9  Q8X10  Q8X11  Q8X12  Q8X13 
Q8X14  Q8X15  Q8X16  Q8X17  Q8X18  Q8X19  Q9X10  Q9X11  Q9X12  Q9X13  Q9X14 
Q9X15  Q9X16  Q9X17  Q9X18  Q9X19  QlOXll  Q10X12  Q10X13  Q10X14  Q10X15 
Q10X16  Q10X17  Q10X18  Q10X19  Q11X12  Q11X13  Q11X14  Q11X15  Q11X16 
Q11X17  Q11X18  Q11X19  Q12X13  Q12X14  Q12X15  Q12X16  Q12X17  Q12X18 
Q12X19  Q13X14  Q13X15  Q13X16  Q13X17  Q13X18  Q13X19  Q14X15  Q14X16 
Q14X17  Q14X18  Q14X19  Q15X16  Q15X17  Q15X18  Q15X19  Q16X17  Q16X18 
Q16X19  Q17X18  Q17X19  Q18X19  SQXl  SQX2  SQX3  SQX4  SQX5  SQX6  SQX7 
SQX8  SQX9  SQXIO  SQXll  SQX12  SQX13  SQX14  SQX15  SQX16  SQX17  SQX18 
SQX19/SLE=.l  SLS=.l; 
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PROC  UNIVARIATE  DATA=PRETEST  NORMAL  PLOT; 

VAR  PFQSl  PFQS2  PFQS3  PFQS4  PFQS5  PFQS6  PFQS7  PFQS8  PFQS9 
PFQSIO  PFQSll  PFQS12  PFQS13  PFQS14  PFQS15  PFQS16  PFQS17  PFQS18 
PFQS19  Q1X2  Q1X3  Q1X4  Q1X5  Q1X6  Q1X7  Q1X8  Q1X9  QIXIO  QlXll  Q1X12 
Q1X13  Q1X14  Q1X15  Q1X16  Q1X17  Q1X18  Q1X19  Q2X3  Q2X4  Q2X5  Q2X6 
Q2X7  Q2X8  Q2X9  Q2X10  Q2X11  Q2X12  Q2X13  Q2X14  Q2X15  Q2X16  Q2X17 
Q2X18  Q2X19  Q3X4  Q3X5  Q3X6  Q3X7  Q3X8  Q3X9  Q3X10  Q3X11  Q3X12  Q3X13 
Q3X14  Q3X15  Q3X16  Q3X17  Q3X18  Q3X19  Q4X5  Q4X6  Q4X7  Q4X8  Q4X9 
Q4X10  Q4X11  Q4X12  Q4X13  Q4X14  Q4X15  Q4X16  Q4X17  Q4X18  Q4X19  Q5X6 
Q5X7  Q5X8  Q5X9  Q5X10  Q5X11  Q5X12  Q5X13  Q5X14  Q5X15  Q5X16  Q5X17 
Q5X18  Q5X19  Q6X7  Q6X8  Q6X9  Q6X10  Q6X11  Q6X12  Q6X13  Q6X14  Q6X15 
Q6X16  Q6X17  Q6X18  Q6X19  Q7X8  Q7X9  Q7X10  Q7X11  Q7X12  Q7X13  Q7X14 
Q7X15  Q7X16  Q7X17  Q7X18  Q7X19  Q8X9  Q8X10  Q8X11  Q8X12  Q8X13  Q8X14 
Q8X15  Q8XI6  Q8X17  Q8X18  Q8X19  Q9X10  Q9X11  Q9X12  Q9X13  Q9X14  Q9X15 
Q9X16  Q9X17  Q9X18  Q9X19  QlOXll  Q10X12  Q10X13  Q10X14  Q10X15  Q10X16 
Q10X17  Q10X18  Q10X19  Q11X12  Q11X13  Q11X14  Q11X15  Q11X16  Q11X17 
Q11X18  Q11X19  Q12X13  Q12X14  Q12X15  Q12X16  Q12X17  Q12X18  Q12X19 
Q13X14  Q13X15  Q13X16  Q13X17  Q13X18  Q13X19  Q14X15  Q14X16  Q14X17 
Q14X18  Q14X19  Q15X16  Q15X17  Q15X18  Q15X19  Q16X17  Q16X18  Q16X19 
Q17X18  Q17X19  Q18X19  SQXl  SQX2  SQX3  SQX4  SQX5  SQX6  SQX7  SQX8 
SQX9  SQXIO  SQXll  SQX12  SQX13  SQX14  SQX15  SQX16  SQX17  SQX18; 

TITLE  ’NORMALITY  TEST’; 

PROC  ANOVA  DATA=PRETEST; 

CLASS  PFQSl  PFQS2  PFQS3  PFQS4  PFQS5  PFQS6  PFQS7  PFQS8  PFQS9 
PFQSIO  PFQSll  PFQS12  PFQS13  PFQS14  PFQS15  PFQS16  PFQS17  PFQS18 
PFQS19; 

MODEL  JOBKNO=PFQSl  PFQS2  PFQS3  PFQS4  PFQS5  PFQS6  PFQS7  PFQS8 
PFQS9  PFQSIO  PFQSll  PFQS12  PFQS13  PFQS14  PFQS15  PFQS16  PFQS17 
PFQS18  PFQS19; 

TITLE  ’ANALYSIS  OF  VARIANCE  FOR  JOB  KNOWLEDGE’; 


run; 
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Appendix  E:  Enlisted  Evaluation  Performance  System  Survey 
USAF  Survey  Control  Number 
GENERAL  INFORMATION 


The  purpose  of  this  questionnaire  is  to  obtain  data  concerning  your  perceptions  of  the 
enlisted  evaluation  performance  feedback  system.  An  effective  performance  appraisal  system 
can  significantly  contribute  to  an  organization’s  productivity.  Therefore,  it  is  in  the  best 
interest  of  the  Air  Force  to  have  an  effective  performance  appraisal  system.  The  results  from 
this  questionnaire  will  be  used  for  data  purposes  only.  Personal  information  will  be  used  for 
classification  purposes  only. 


INSTRUCTIONS 


Please  use  the  enclosed  AFIT  DATA  COLLECTION  FORM  when  filling  out  this  survey.  If 
the  collection  form  is  lost  or  is  damaged,  please  write  your  answers  on  the  questionnaire  and 
return  it  instead  of  the  AFIT  Data  Collection  Form. 

-  This  survey  is  strictly  anonymous. 

Some  rules  to  remember  when  filling  out  this  form: 

•  Use  only  a  number  2  pencil. 

•  Make  dark  marks  that  fill  in  the  circle  completely. 

•  Erase  clearly  any  mark  you  wish  to  change. 

•  Make  no  stray  marks. 

•  DO  NOT  fold  the  data  collection  form. 

-  DO  NOT  put  your  name  anywhere  on  the  data  collection  form. 

-  DO  NOT  put  your  social  security  account  number  anywhere  on  the  data  collection 
form. 

-  After  you  have  filled  out  the  data  collection  form,  put  the  form  and  the  survey 
questionnaire  in  the  enclosed  return  envelope,  seal  the  envelope,  and  mail  it  to  us.  Note:  The 
postage  on  the  return  envelope  is  prepaid  and  the  return  address  already  appears  on  the 
envelope. 
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Fill  out  the  rest  of  the  data  odlection  form  be^nning  with  question  001  (question  001  is 
located  on  the  page  which  is  marked  AFTT  DATA  COLLECTION  FORM). 

(^estionnaire 


001.  My  rank  is 

1.  El,E2,orE3 

2.  E4 

3.  E5 

4.  E6 

002.  MyMAJCOMis 

1.  ATC 

2.  AFMC 

3.  ACC 

4.  Other 

003.  My  skill  level  is 

1.  One  Level 

2.  Three  Level 

3.  Five  Level 


5. E7 

6.  E8 

7. E9 


4.  Seven  level 

5.  Nine  Level 
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004.  My  total  number  of  years  on  active  duty  is 

1.  less  than  1. 

2.  more  than  1  but  less  than  4 

3.  more  than  4  but  less  than  10 

4.  more  than  10  but  less  than  IS 

5.  more  than  15  but  less  than  20 

6.  more  than  20  but  less  than  26 

7.  more  than  26 

Note:  If  your  total  number  of  years  on  active  duty  is  exactly  one  year,  you  should  respond 
with  answer  2.  If  your  total  number  of  years  on  active  duty  is  exactly  4  years,  you  should 
respond  with  answer  3,  and  so  on. 

005.  My  highest  PME  is  the 

1.  NCXD  Preparatory  Course  (NCOPC) 

2.  NCO  Leadership  School  (NCOLS) 

3.  NCO  Academy  (NCOA) 

4.  Senior  NCO  Academy  (SNCOA) 

5.  I  have  never  completed  a  PME  course. 

006.  My  most  recent  PME  was  completed  by 

1.  Correspondence 

2.  In-residence 

3.  I  have  never  completed  a  PME  course. 
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007.  My  most  recent  PME  was  completed 

1.  less  than  a  month  ago 

2.  1-6  months  ago 

3.  6-12  months  ago 

4.  more  than  12  months  ago 

5.  I  have  never  completed  a  PME  course. 

008.  My  most  recent  performance  feedback  session  (as  the  ratee)  was  conducted 

1.  less  than  a  month  ago 

2.  1-6  months  ago 

3.  6-12  months  ago 

4.  more  than  12  months  ago 

5.  I  have  never  received  performance  feedback 

009.  The  last  time  you  received  performance  feedback  was 

the  session  documented  on  the  Performance  Feedback  Worksheet  (PFW)? 

1.  Yes 

2.  No 

3.  I  have  never  received  performance  feedback 
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Please  choose  the  number  on  the  scale  that  best  describes  your  opinion  of  these 
statements. 


ill  Ik  :|i  4i  *  i|c  lit  :|t  Ik  lit  *  111  i|t  ii>  ]|c  i|t  4i  i|c  i|t  4i  ]|i  111  i|<  ik  ill  it>  41 <■  >!■  41  >i>  >i"l>  Ik  itt  *  *  41  *  *  )|<  ii>  iltik  >i>  iti  *  *  Ik  *  i|<  i|<  III  *  ♦ 


Strongly 

Disagree 

i 

Disagree 

1 

Slightly 

Disagree 

1 

Neither 
Agree  nor 

Disagree 

\ 

Slightly 

Agree 

1 

Agree 

1 

Stongly 

Agree 

1 

1 

2 

3 

4 

5 

6 

7 

4iik4i4>4i4i4i4i4i4>4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4>4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4‘4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4i4c 


When  marking  your  data  oriiection  form,  1  (Strongly  Disagree)  corresponds  to  1,  2 
(Disagree)  corresponds  to  2,  and  so  on. 

010.  I  understand  my  responsibilities  as  an  airman/NCO. 

Oil.  I  feel  my  current  skill  level  is  commensurate  with  my  capacity  to  accomplish  my  job. 
012.  My  obligations  as  an  airman/NCO  are  clear  to  me. 

013.  I  have  a  clear  understanding  of  the  role  I  play  in  accomplishing  the  wing’s  mission. 

014,  My  skill  level  does  not  do  a  good  job  of  measuring  my 
ability  to  do  my  job. 

015.  I  understand  the  importance  of  my  position  within  the  squadron. 

016.  My  job  proficiency  is  accurately  reflected  by  my  skill  level. 

017.  I  understand  the  importance  of  my  position  within  the  wing. 

018.  The  way  in  which  my  work  impacts  the  effectiveness  of  the  wing  is  not  clear  to  me. 

019.  I  realize  the  importance  of  my  job  with  respect  to  accomplishing  the  goals  of  my 
squadron. 

020.  I  do  not  understand  my  duties  as  an  airman/NCX). 

021.  The  way  in  which  my  work  impacts  the  effectiveness  of  the  squadron  is  not  clear  to 
me. 


If  you  have  never  received  performance  feedback,  you  may  skip  to  the  last  three 
questions  of  the  survey. 
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Please  choose  the  number  on  the  scale  that  best  describes  your  opinion  of  your  most 
recent  p^ormance  feedbadc  session. 

*:tt4,itntt**itHt:t‘*^t‘****************************************************************** 


Strongly 

Disagree 

1 

Disagree 

1 

Slightly 

Disagree 

1 

Neither 
Agree  nor 

Disagree 

1 

Slightly 

Agree 

1 

Stongly 

Agree  Agree 

1  1 

1 

2 

3 

4 

5 

6  7 

When  marking  your  data  collectimi  form,  1  (Stitmgly  Disagree)  corresponds  to  1,  2 
(Disagree)  corresponds  to  2,  and  so  on. 

022.  The  performance  feedback  session  covered  my  entire  job. 

023.  The  discussion  of  my  performance  during  the  feedback  session  was  covered  objectively. 
024.  The  feedback  session  was  accurately  conducted. 

025.  I  didn’t  have  to  ask  for  clarification. 

026.  The  feedback  session  was  fair  in  every  respect. 

027.  The  feedback  session  really  raised  my  anxiety  level. 

028.  The  purpose  of  the  feedback  session  was  not  clear  to  me. 

029.  The  feedback  session  really  made  me  think  about  working  smarter  on  the  job. 

030.  The  feedback  session  was  encouraging  to  me  personally. 

031.  I  dreaded  the  actual  feedback  session  itself. 

032.  The  supervisor  was  straightforward  in  all  phases  of  the  feedback  session. 

033.  The  feedback  session  gave  me  some  direction  and  purpose. 

034.  The  feedback  session  really  pinpointed  areas  for  improvement. 

035.  The  feedback  session  was  frustrating. 

036.  The  feedback  session  was  disorganized. 

037.  I  disliked  the  feedback  session  because  the  intent  was  not  clear. 
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Please  choose  the  number  on  the  scale  that  best  describes  your  opinimi  of  your  most 
recent  performance  feedback  session. 


Strongly 

Disagree 

1 

Disagree 

1 

Slightly 

Disagree 

1 

Neither 
Agree  nor 

Disagree 

1 

Slightly 

Agree 

1 

Stongly 

Agree  Agree 

1  1 

1 

2 

3 

4 

5 

6  7 

*ltt!|tl|t*it*>|l****^l*^l*^l*!t‘*************************^t***^^***********^t^t**^t**)^l***^^**:^i*)^t* 


When  marking  your  data  ctdlection  form,  1  (Strongly  Disagree)  corresponds  to  1,  2 
(Disagree)  corresponds  to  2,  and  so  on. 

038.  The  supervisor  who  conducted  the  feedback  session  was  not  well  trained. 

039.  The  feedback  session  has  been  my  guide  for  correcting  weaknesses. 

040.  I  understood  the  meaning  of  each  performance  area  better  after  the  feedback  session. 
041.  The  feedback  session  was  too  rushed. 

042.  I  received  no  advance  notice  about  the  feedback  session. 

043.  During  the  feedback  session,  my  performance  was  analyzed  fairly. 

044.  I  was  often  upset  because  the  feedback  session  data  were  inaccurate. 

045.  My  record,  as  it  was  introduced  in  the  feedback  session,  contained  no  errors. 

046.  I  was  worried  that  the  feedback  session  worksheet  would  not  be  kept  contidential. 

047.  Specific  examples  of  my  performance  were  cited. 

Questions  48  and  49  should  be  answered  on  the  page  of  the  data  ctdlection  form  which  is 
marked  YOUR  WORK  GROUP  CODE. 

048.  Fill  in  the  circles  corresponding  to  your  Air  Force  Specialty  (Itode  (in  the  section  titled 
"YOUR  AFSC"). 

049.  Fill  in  the  circle  corresponding  to  your  answer  to  the  question  "ARE  YOU  A 
SUPERVISOR?". 

050.  If  you  have  any  additional  comments  about  the  enlisted  performance  feedback  system  or 
this  questionnaire,  please  write  them  on  the  back  of  this  page. 


Thank  You  for  Your  Participation 
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